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CHAPTER  I 


INTRODUCTION 

Background 

Recent  Fighter  Aircraft 
Experience 

U.S.  Air  Force  tactical  aircraft  of  the  1960s  and 
early  1970s  proved  their  worth  and  capabilities  during 
the  Southeast  Asian  conflict.  The  experience  gained  in 
this  actual  combat  setting  proved  valuable  in  discriminating 
between  those  characteristics  desirable  in  a  tactical 
fighter  and  those  factors  which  degraded  overall  mission 
effectiveness.  Although  for  the  most  part  they  had  been 
designed  to  perform  specific  missions,  Vietnam-era  fighter 
aircraft  were  employed  in  a  variety  of  roles.  The  natural 
progression  in  design  of  these  aircraft  from  earlier  vari¬ 
ants  had  seen  them  grow  heavier,  faster  (normally),  and 
more  complex  than  their  predecessors.  The  rapid  advance 
of  technology  was  most  evident  in  the  avionics  and  fire  con¬ 
trol  radars  of  these  aircraft. 

The  F-4,  as  a  multi-role  aircraft,  represented  an 
excellent  compromise  of  the  mission  demands  placed  upon  it. 
To  maintain  its  combat  effectiveness,  the  Air  Force 
enhanced  its  capabilities  through  evolutionary  modification 
in  the  form  of  increased  thrust,  improved  fire  control 
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radar,  and  electro-optical  ordinance  delivery  systems.  The 
F-4 ,  however,  became  less  effective  in  some  aspects  of  its 
mission  demands  due  to  its  age,  increased  gross  weight  and 
declining  relative  performance  characteristics.  It  became 
apparent  that  there  was  a  need  for  a  multi-role  combat 
fighter  in  the  1970s  to  replace  the  existing  fleet.  Air 
Force  requirements  at  this  time  envisioned  an  aircraft 
that  could  perform  conventional  weapons  delivery  in  addi¬ 
tion  to  fulfilling  the  tactical  nuclear  and  air  defense 
postures . 

Evolution  and  Development  of 
the  Air  Combat  Fighter 

The  U.S.  Air  Force  lightweight  fighter  program 
that  began  in  1972  had  the  initial  purpose  of  investigating 
new  technology  that  might  be  appropriate  to  future  U.S. 
fighter  aircraft  requirements  (6:34).  In  addition  to  demon¬ 
strating  advanced  technology,  the  lightweight  fighter  pro¬ 
gram  aimed  to  reduce  technical  and  cost  uncertainties 
involved  in  the  development  of  fighter  aircraft.  In  an 
attempt  to  motivate  contractor  innovation  through  competi¬ 
tion,  the  USAF  selected  two  companies,  General  Dynamics 
and  Northrop,  to  design  and  build  two  prototypes.  When 
the  USAF  determined  that  such  a  concept  was  feasible  and 
readily  functional,  officials  accelerated  the  prototype 
program  to  a  competitive  flyoff  (15:126). 
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The  program  exhibited  a  central  change  from  conven¬ 
tional  management  approaches  in  that  the  contract  set  only 
desired  performance  goals  without  the  rigidity  of  contrac¬ 
tual  specifications  (17:50).  USAF  philosophy,  then,  was 
to  manage  the  program  only  by  exception  and  to  allocate 
the  routine  decisions  to  the  contractors.  In  fact,  USAF 
officials  gave  full  design  responsibility  to  the  con¬ 
tractors,  a  measure  which  was  meant  to  convey  the  govern¬ 
ment's  willingness  to  accept  and  evaluate  new  ideas,  to 
recognize  the  risk  associated  with  technological  advance, 
and  to  free  the  contractors  from  unnecessary  constraints 
(17:50)  . 

The  lightweight  fighter  program,  under  which  the 
General  Dynamics  YF-16  and  Northrop  YF-17  developed,  took 
on  greater  significance  when  in  April  1974  the  Secretary 
of  Defense  announced  that  the  Pentagon  was  considering  full- 
scale  development  of  an  air  combat  fighter  (ACF)  derivative 
of  either  the  YF-16  or  YF-17  (17:51).  The  lightweight 
fighter  was  am  obvious  choice  for  the  air  combat  fighter 
because  of  the  increasing  complexity  and  resulting  costs 
associated  with  more  sophisticated  weapon  systems. 

During  this  period,  the  forecast  Soviet  threat 
depicted  the  USSR  overtaking  the  U.S.  in  quality  as  well 
as  quantity  of  fighter  aircraft.  It  was  important  then, 
that  the  USAF  maintain  a  force  large  enough  to  meet  its 
current  commitments  in  addition  to  modernizing  the  fighter 
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fleet.  The  philosophy  at  the  time  advocated  that  the 
simplest  weapon  system  possible  which  could  accomplish  a 
mission  should  be  procured  in  order  to  insure  the  lowest 
acquisition  cost  along  with  the  lowest  operating  and  sup¬ 
port  costs  (18:89). 

Each  prototype  was  designed  for  the  visual  air  com¬ 
bat  environment  only,  a  move  that  was  to  minimize  life- 
cycle  costs  by  requiring  less  avionics  (18:77).  Both  proto 
type  aircraft  had  been  designed  to  a  target  cost  of 
$3,000,000  per  unit  in  fiscal  year  1972  dollars,  based  on 
an  assumed  production  run  of  300  aircraft  consisting  of 
the  airframe,  engine,  heads-up  display,  gun  and  simple 
ranging  device  (17:50). 

As  a  result  of  the  competitive  flyoff,  cost  and  mis 
sion  considerations,  the  USAF  selected  the  General  Dynamics 
YF-16  as  the  winner  of  the  air  combat  fighter  competition, 
and  the  corporation  was  awarded  the  full-scale  development 
contract  in  January  1975. 

Fire  Control  System  for  the  F-16 

Evolution.  In  1975,  when  full-scale  development 
began,  the  Tactical  Air  Command  (TAC)  did  not  have  a 
required  operational  capability  (ROC)  prepared  for  the 
F-16  (19:7).  Although  the  F-16  was  conceived  as  a  replace¬ 
ment  for  the  F-4 ,  it  originally,  as  the  ACF,  was  forecast 
to  perform  in  the  visual  air  combat  arena  alone.  TAC 


revised  the  F-16  mission  employment  roles,  however,  which 
dictated  that  a  more  complex  and  comprehensive  fire  control 
radar  (FCR)  be  developed,  a  system  which  would  have  radar- 
missile,  ground  mapping  and  electro-optical  weapons 
delivery  capabilities. 

The  Air  Force  decided  that  the  enhanced  F-16  fire 
control  radar  could  be  built  as  contractor-furnished  equip¬ 
ment  (CFE)  in  accordance  with  TAC  requirements.  One  of  the 
fire  control  radars  submitted  as  a  subcontractor  proposal, 
the  Westinghouse  WX-200,  was  an  indirect  descendant  of  the 
F-4  APQ-120  radar.  In  November  1975,  General  Dynamics 
announced  the  selection  of  Westinghouse  as  subcontractor 
for  the  F-16  radar  (18:77). 

The  newly  designated  APG-66  radar  pressed  the 
DSARC  I I IB  (full  production  decision)  deadline  of  September 
1977  when  pre-production  testing  fell  behind  schedule. 

Prior  to  the  full  production  decision,  the  subcontractor 
had  to  successfully  demonstrate  all  radar  functions  and 
the  integration  of  the  radar  with  the  other  F-16  avionics 
subsystems.  A  flight  model  of  the  F-16  radar  had  demon¬ 
strated  its  capability  to  perform  most  of  the  required 
functions  while  flown  in  an  F-4  testbed;  however,  the  set 
was  20  percent  larger  than  the  one  to  be  used  in  the  produc¬ 
tion  aircraft  (19:7).  The  ensuing  technical,  reliability 
and  maintainability  development  of  the  system  is  discussed 
in  detail  and  analyzed  in  Chapter  IV. 


5 


APG-66  System  Description.  The  APG-66  fire  con¬ 
trol  radar  for  the  F-16  is  a  coherent,  multi-mode,  digital 
fire  control  sensor  designed  to  complement  the  air  super¬ 
iority  and  strike  roles  of  the  aircraft  (23:vii).  The  sys¬ 
tem  was  designed  with  digital  technology  and  sophisticated 
software  to  enable  ease  of  maintenance  and  to  provide  a 
more  reliable  product. 

The  design  features  several  maintainability  consider¬ 
ations  including  the  absence  of  blind  connectors  and  the 
fact  that  no  flightline  adjustments  or  special  tools  are 
required.  Six  functional  line  replaceable  units  (LRU) 
which  are  organized  for  autonomy,  function,  minimum  inter¬ 
connection,  and  ease  of  maintenance  compose  the  system 
(23:vii).  The  six  line  replaceable  units  are  the  antenna, 
transmitter,  low-power  radio  frequency  unit  (LPRF) ,  digital 
signal  processor  (DSP),  the  radar  computer,  and  a  radar 
control  panel.  A  rack  assembly  is  mounted  to  the  aircraft 
which  retains  the  transmitter,  DSP,  computer,  LPRF  and 
associated  cabling  and  waveguide.  All  of  the  radar  LRUs 
are  mounted  in  the  nose  of  the  aircraft  and  are  accessible 
from  ground  level,  except  for  the  radar  control  panel 
mounted  in  the  cockpit. 

The  system  has  built- in-test  (BIT)  and  self-test 
capabilities  that  both  alert  the  user  of  degraded  system 
functions  and  provide  maintenance  personnel  with  a  detailed 
history  and  description  of  fire  control  radar  failures. 
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The  built-in  tests  require  maintenance  personnel  or  pilot 
participation  in  order  to  detect  or  isolate  a  system  fault 
to  a  particular  LRU  (23:vii).  Self-tests  are  character¬ 
ized  as  automatic  noninterferring  performance  testing  in 
which  either  continuous  or  iterative  monitoring  techniques 
may  be  applied.  A  fire  control  computer  operational  flight 
program  is  integrated  to  collect  and  store  faults  from  the 
radar,  to  command  avionic  equipments  to  perform  detailed 
self-tests,  and  to  provide  a  display  of  the  results  of  the 
self-tests  to  the  pilot  or  maintenance  personnel  (23:vii). 

Statement  and  Scope  of  the  Problem 
Problem  Statement 

The  armed  services  strive  to  optimize  the  acquisi¬ 
tion  of  weapon  systems  in  order  to  provide  the  most  effec¬ 
tive  national  defense  for  the  appropriated  defense  budget. 
One  of  the  paramount  factors  in  any  major  weapon  system 
acquisition  is  the  minimization  of  future  system  life- 
cycle  costs.  The  reliability  and  maintainability  that  the 
system  exhibits  in  its  deployed  phase  directly  determines 
the  amount  of  these  costs.  In  addition,  reliability  and 
maintainability  determines  the  effectiveness  of  several 
critical  measures  of  defense  readiness  including  system 
availability,  unit/individual  combat  readiness,  and  mission 
effectiveness.  Although  the  defense  services  place  a  great 
deal  of  emphasis  on  reliability  and  maintainability  during 
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weapon  systems  acquisition,  an  optimal  approach  has  not 
been  found.  As  a  recent  major  acquisition,  the  APG-66 
development  displayed  various  management  and  design  mea¬ 
sures  indicative  of  a  concentrated  reliability  and  main¬ 
tainability  effort.  This  thesis  will  examine  reliability 
and  maintainability  of  the  F-16  APG-66  fire  control  radar 
in  comparison  with  its  contractual  requirements  and  the 
operational  results  of  the  APQ-120  radar  which  was  its  pre¬ 
decessor  . 

Scope 

This  thesis  analyzes  the  APG-66  reliability  man¬ 
agement  history  in  the  critical  design  review  and  reliabil¬ 
ity  qualification  test  phases  of  development.  No  attempt 
will  be  made  to  determine  the  applicability  of  test  specifi 
cations  or  MIL- STD  781B  procedures.  Operational  field 
reliability  data  used  to  ascertain  reliability  growth  and 
other  measures  of  reliability  comparison  are  gathered  via 
AFM  66-1  (Aircraft  Maintenance  Management)  maintenance  data 
reports  on  USAF-owned  continental-U.S.  based  aircraft  only. 
This  data  is  assumed  to  be  valid.  No  attempt  is  made  to 
analyze  the  European  consortium  aircraft  procurement  or 
their  reliability  performance. 

Analysis  of  the  APG-66  maintainability  management 
history  is  limited  to  an  examination  of  the  critical  design 
review  and  the  intermediate  level  maintainability 
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demonstration  in  comparison  with  a  twenty-month  window  of 
operational  results. 

A  major  consideration  in  this  effort  is  the  phase 
of  APG-66  system  maturity  in  comparison  with  the  APQ-120 
fire  control  radar.  In  some  analyses,  the  data  presented 
for  the  prior  generation  system  represents  a  period  that 
is  three  years-plus  from  the  completion  of  reliability 
qualification  testing.  Although  the  data  management  of 
system  failure  reporting  has  changed  since  the  1973  study 
of  APQ-120  reliability,  this  effort  attempts  to  present 
both  data  sources  on  a  common  normalized  base. 

Objectives 

The  main  objective  of  this  thesis  effort  is  to 
determine  the  effectiveness  of  the  APG-66  subcontractor's 
reliability  test  program  and  maintainability  plan  in  pro¬ 
ducing  a  reliable  and  maintainable  weapon  subsystem.  A 
major  subobjective  is  to  determine  the  relative  increase 
in  fire  control  radar  reliability  in  comparison  with  a 
representative  prior  generation  radar. 

As  a  case  study,  the  thesis  will  attempt  to  aid  sys¬ 
tems  program  office  personnel  in  the  acquisition  of  future 
weapon  systems. 
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CHAPTER  II 


LITERATURE  REVIEW 

Reliability  and  Maintainability 

Definition  and  Relationship 
to  Life-Cycle  Cost 

Igor  Bazovsky  in  1961  noted  the  impact  of  reliabil¬ 
ity  with  respect  to  production,  waste,  technology,  and 
safety: 

The  lack  of  reliability  wastes  billions  of  dollars 
and  has  slowed  technological  progress  in  many  vital 
areas.  There  is  perhaps  no  engineering  field  today 
where  the  need  for  improvement  is  greater  than  in  the 
field  of  reliability  [3:vii]. 

Reliability  is  not  a  theoretical  notion  without  practical 

application;  rather  it  .  .is  quantitive  in  concept, 

predictable  in  design,  measureable  in  test,  assurable  in 

production,  and  maintainable  in  the  field  [14:4.6]." 

Reliability,  in  general  terms,  can  be  defined  as:  ",  .  , 

the  probability  that  a  system  can  meet  an  operational 

requirement  for  a  given  period  when  the  system  operates 

under  certain  conditions  [14:4.3]."  The  importance  of 

reliability  in  weapon  systems  cannot  be  overstated. 

Greater  reliability  provides  for  greater  availability  of 

necessary  weapon  systems  and,  more  importantly,  it  provides 

an  increased  probability  that  these  weapon  systems  can 

successfully  complete  their  missions. 


The  most  direct  method,  theoretically  and  prac¬ 
tically,  to  achieve  reliability  in  weapon  systems  is  to 
design  and  produce  simple  systems.  While  desirable, 
simplicity  in  design  is  not  a  feasible  approach  in  the 
development  of  some  defense  hardware.  A  prime  example  is 
aircraft  avionics,  where  improved  capability  is  usually 
accomplished  through  technological  advances  which  require 
more  complex  systems.  Inherently  then,  mission  requirements 
for  improved  technical  performance  in  avionics  and  fire  con¬ 
trol  radars  often  result  in  reduced  reliability. 

The  fact  that  a  declining  defense  procurement  budget 
buys  fewer  increasingly  complex  systems  emphasizes  the  role 
of  system  reliability  in  the  determination  of  weapon  system 
life-cycle  costs.  A  weapon  system  experiences  three  dis¬ 
tinct  phases  of  failure  during  its  life  cycle: 

1.  Burn-in  (early  life)  period.  During  this  time 
manufacturing  practices  are  refined,  product  tolerances 
are  reduced,  and  improvement  in  designs  are  made.  Testing 
during  this  period  improves  system  performance  so  that  the 
burn-in  period  diminishes  rapidly  and  reaches  the  design 
level  (14:4. 10) . 

2.  Useful  life  period.  During  this  period,  sys¬ 
tems  experience  chance  failures  according  to  one  of  several 
probability  distributions,  normally  the  exponential.  In 
addition,  during  this  period,  systems  reach  their  lowest 
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failure  rate,  and  thus  they  can  be  utilized  to  the  great¬ 
est  advantage  (3:33). 

3.  Wearout  period.  During  this  period,  systems 
experience  increased  failure  over  that  experienced  during 
useful  life.  High  system  reliabilities  for  extended  periods 
of  time  can  be  achieved  by  periodic  replacement  of  com¬ 
ponents  before  they  fail  and  periodic  inspections  at  spe¬ 
cific  intervals  (3:36). 

The  relationship  of  reliability  to  weapon  system  life- 
cycle  cost  i,s  a  direct  one.  Bazovsky  (3:171)  determined 
that  improvements  in  system  reliability  can  best  be  made, 
and  at  the  lowest  cost,  in  the  design  of  the  weapon  system 
through  tradeoffs  in  system  performance .  Total  acquisition 
costs  increase  as  the  reliability  improvement  decision  is 
delayed  until  later  phases  of  the  system  life  cycle.  When 
a  weapon  system  is  operationally  deployed  and  requires 
reliability  modifications,  not  only  are  costs  higher,  but 
the  system  experiences  lower  availability.  Thus,  it  is 
desirable  to  have  intensive  research  and  reliability 
engineering  in  the  design  phases  rather  than  after  a 
weapon  system  is  in  the  operational  inventory. 

Department  of  Defense  Reliability 
Experience  and  Emphasis  " 

The  high  financial  costs  of  unreliable  systems  are 
not  of  recent  discovery.  In  the  mid-1960s  a  pioneer  effort 
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was  made  to  estimate  these  costs  to  the  Royal  Air  Force 
(RAF) .  The  report  estimated  that  failure  of  equipment  to 
live  up  to  its  expected  performance  cost  the  RAF  approxi¬ 
mately  40  percent  of  its  annual  budget  (13:23). 

U.S.  Department  of  Defense  (DOD)  officials  recog¬ 
nized  the  effects  reliability  and  maintainability  have  on 
system  life-cycle  costs  (LCC) ,  mission  effectiveness  and 
defense  readiness.  In  order  to  emphasize  DOD  complicity, 

DOD  Directive  5000.40  (Policy  Statement  on  Reliability) 
officially  issued  guidelines  that  all  DOD  program  managers 
must  follow  to  ensure  that  reliability  and  maintainability 
factors  are  included  in  all  design  efforts  (5:56). 

Defense  efforts  to  introduce  lower  cost  weapon  sys¬ 
tems  that  are  more  reliable,  available,  and  maintainable 
have,  however,  been  less  than  fully  successful.  A  report 
on  the  implications  of  highly  sophisticated  weapon  systems 
on  military  capabilities  was  performed  in  June  1980  by  the 
U.S.  General  Accounting  Office  (GAO).  The  review  was 
requested  by  the  Chairman  of  the  Senate  Committee  on  Foreign 
Relations  as  the  result  of  a  1979  study  on  impediments  to 
reducing  the  costs  of  weapon  systems.  The  GAO  report  con¬ 
cluded  that  despite  increased  DOD  efforts  in  reliability 
and  maintainability,  the  results  fell  short  of  desired 
goals.  The  report  also  predicted  that  weapon  systems 
designed  in  the  mid-1970s  are  likely  to  experience  many  of 
the  same  problems  that  occurred  in  high  performance  weapons 
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deployed  at  the  time  of  the  study  (18 : Abstract ) .  The  review 
recommended  that  personnel  involved  with  the  acquisition 
of  costly  systems  must  be  fervently  concerned  with  the 
deployed  equipment's  field  capability,  and  also  suggested 
that  increased  emphasis  should  be  made  in  the  early  design 
of  weapon  systems  in  order  to  obtain  an  optimum  mix  of  high 
performance  and  support  characteristics  (18 :Abstract) . 

Current  Department  of  Defense  directives  require 
that  LCC  estimates  be  formulated  in  the  validation  phase 
and  reviewed  at  each  Defense  System  Acquisition  Review 
Council  ( DS ARC )  milestone.  With  700  different  LCC  models 
to  choose  from,  the  cost  estimating  process  is  not  a  simple 
one.  Compounding  the  estimating  problem  is  the  fact  that 
equipment  hardware  as  well  as  the  acquisition  plan  goes 
through  metamorphosis  in  this  phase.  In  an  attempt  to 
balance  mission  performance  requirements  against  the  costs 
of  unreliable  systems,  some  recent  acquisition  strategies 
favored  acquiring  those  systems  that  performed  acceptably 
and  had  the  lowest  life-cycle  cost  (4:99). 

Focusing  attention  on  the  reliability  variable  of 
a  life-cycle  cost  model  does  not  necessarily  result  in 
lower  acquisition  and  operational  and  support  costs,  how¬ 
ever.  Air  Force  Systems  Command  personnel  in  a  1976  inter¬ 
view  determined  several  drawbacks  of  the  reliability- 
oriented  approach  (9:22): 
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1.  If  technically  and  operationally  feasible, 
high-spares/maintenance  cost  subsystems  or  components 
should  be  eliminated  which  would  result  in  more  savings 
than  improving  their  reliability  would  accomplish.  (The 
key  to  this  approach  is  the  fact  that  Warsaw  Pact  state-of- 
the-art  technology  in  deployed  weapons  is  rapidly  approach¬ 
ing  that  of  the  U.S.  This  makes  it  necessary  for  the  U.S. 
to  incorporate  their  newest  technology  in  the  design  and 
production  of  their  aircraft.  The  inherent  reliability 
associated  with  this  new  technology  will  be  somewhat  less 
than  if  tried  and  proven  components  were  utilized.) 

2.  Reliability  and  maintainability  improvement 
must  be  coordinated  with  an  effort  to  reduce  manpower  when 
that  system  is  deployed.  If  the  operating  commander  does 
not  have  the  option  of  reducing  his  maintenance  personnel 
roster,  the  costs  will  still  exist. 

Reliability  Test  Program 

The  contractor's  reliability  test  program  is  one 
of  three  reliability  elements  considered  basic  to  the 
achievement  of  equipment  reliability .  .This  element  is  com¬ 
prised  of  three  phases;  environmental  screening,  evaluation 
testing,  and  qualification  testing  (8:7). 

Environmental  screening  is  conducted  at  the  equip¬ 
ment  level  during  system  development  in  order  to  remove 
infant  or  early  mortality  failures  from  the  equipment. 
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This  phase  serves  several  purposes;  it  removes  failures 
normally  associated  with  poor  workmanship  by  parts  vendors, 
and  it  determines  if  the  equipment  is  capable  of  meeting 
its  predicted  performance  parameters  when  subjected  to 
simulated  field  use  environments. 

The  second  phase  of  the  test  program,  evaluation 
testing,  has  the  intended  purpose  of  revealing  design  and 
manufacturing  deficiencies  that  cannot  be  detected  during 
the  design  of  the  equipment.  The  tests  are  characterized 
by  a  test-and-fix  atmosphere  where  equipment  failures  are 
analyzed,  corrective  actions  are  implemented,  and  the 
effectiveness  of  the  actions  are  evaluated  during  the 
remainder  of  the  test.  Evaluation  testing  is  normally  con¬ 
ducted  at  the  equipment  level  of  development  where  it  is 
subjected  to  temperature  cycling  and  periodic  vibration 
patterned  after  MIL-STD-781.  There  is  considerable  evi¬ 
dence  that  equipment  reliability  grows  exponentially  with 
accumulated  test  time  during  a  properly  conducted  evalua¬ 
tion  test  (8:8) . 

The  reliability  qualification  test  phase  is  the 
final  system  demonstration  prior  to  production.  In  this 
phase,  the  equipment  is  subjected  to  an  environment  dic¬ 
tated  by  MIL-STD-781  in  order  to  measure  the  mean  time 
between  failure  (MTBF)  achieved  as  a  result  of  the  relia¬ 
bility  engineering  program.  The  reliability  qualification 
test  affects  the  final  reliability  of  the  product  in  two 
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ways  (8:8):  it  motivates  the  contractor  to  properly  conduct 
the  developmental  reliability  program,  and  it  allows  an 
assessment  of  corrective  action  effectiveness  prior  to 
full-scale  production. 

Reliability  Studies 

The  problem  of  unreliable  weapon  systems  hardware 
has  been  the  subject  of  intense  research  and  analysis, 
particularly  in  the  defense  community.  This  section  will 
summarize  the  important  findings  of  those  studies  and,  in 
addition,  one  of  the  projects,  a  study  of  APQ-113,-114,-120, 
and  -144  radar  reliability,  will  provide  a  basis  for  a 
comparative  analysis  developed  in  Chapters  IV  and  V. 

Air  Force  experience  with  reliability  in  aircraft 

i 

avionics  has  not  been  promising.  HQ  USAF  recognized  the 
inherent  unreliability  of  aircraft  radars  and  began  an 
effort  to  solve  the  problem.  A  1973  report,  funded  by  the 
USAF,  was  performed  by  the  General  Electric  Aerospace  Elec¬ 
tronic  Systems  Department  in  a  study  of  APQ-113,-114 ,-144 
and  APQ-120  radar  reliability.  The  objective  of  the  study 
was  to  correlate  differences  in  radar  reliability  perform¬ 
ance  compared  to  the  equipment  reliability  requirements  and 
program  structures.  The  study,  which  used  reported  field 
reliability  data,  took  considerable  interest  in  those  fac¬ 
tors  identified  as  major  contributors  of  equipment 
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reliability:  reliability  evaluation  testing,  parts  screen¬ 
ing,  and  equipment  burn-in  (7:iii). 

Two  significant  results  of  the  study  were  among 
those  found  to  be  very  important  in  other  reliability 
research  efforts.  First,  the  report  emphasized  the  impor¬ 
tance  of  uncompromising  contractual  incorporation  of 
MIL-STD-781  at  applicable  airborne  stress  levels  as  the 
principal  driving  force  in  establishing  and  executing 
effective  reliability  development  efforts  (7:iii).  Second, 
the  study  agreed  with  current  acquisition  philosophy  that 
properly  developed  and  directed  reliability  program  invest¬ 
ment  can  produce  significant  cost  savings  leverage  when 
compared  against  the  projected  equipment  life-cycle  main¬ 
tenance  costs  for  unreliable  equipment  (7:iii). 

Also  in  1973,  USAF  contracted  a  study  of  avionics 
equipment  acquisition  in  order  to  develop  basic  relation¬ 
ships  capable  of  determining  and  predicting  the  acquisi¬ 
tion  costs  attributable  to  equipment  reliability.  The 
results  of  the  study  provided  insight  into  the  relationship 
between  reliability  development  cost  and  resulting  relia¬ 
bility  (8:4).  An  analysis  of  ten  contractors’  reliability 
design,  parts  and  test  programs  revealed  interesting  sta¬ 
tistics  reflecting  the  incremental  gain  in  reliability 
MTBF  due  to  the  various  phases  of  the  contractors'  programs. 
Incremental  gains  in  MTBF  due  to  investment  in  the  relia¬ 
bility  design  program  ranged  from  1.4  to  2.9  times  (1.8 
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times  average)  the  initial  of f-the-board  MTBF;  the  gains 
resulting  from  investment  in  the  reliability  parts  program 
ranged  from  2.5  to  4.8  times  (3.4  times  average)  the  design 
gain;  while  the  most  noteworthy  results  were  attained  dur¬ 
ing  the  reliability  test  program  where  the  gains  ranged 
from  2.3  to  35.3  times  (12.8  times  average)  the  parts  gain 
(8  : 18 )  . 

A  1976  Defense  Management  Journal  review  concluded 
that  three  important  factors  emerged  from  various  reliabil¬ 
ity  studies:  the  user  and  contractor  must  maintain  a  close 
planning  and  management  interface ;  complex  multiple  con¬ 
tractual  incentives  can  be  counterproductive;  and  contrac¬ 
tors  must  be  involved  in  the  life-cycle  of  the  product 
(16:52).  A  panacea  does  not  exist  which  could  resolve  the 
reliability  problems  involved  in  weapon  system  acquisition; 
rather,  most  experts  advocate  tailoring  acquisition  strate¬ 
gies  to  each  new  system  as  it  enters  the  conceptual  phase 
(16:53).  Findings  also  revealed  that  although  the  first 
incentive  to  improve  reliability  should  be  exploited  in 
requests  for  design  concept  proposals,  the  effort  must 
continue  through  full-scale  development,  where  in  addition 
to  incentives,  emphasis  can  be  placed  on  tradeoffs  between 
reliability,  operational  performance  and  costs. 

The  disparity  between  predicted  laboratory  relia¬ 


bility  and  operational  field  reliability  is  a  problem  that 
is  frequently  encountered  in  weapon  system  acquisition. 


The  USAF  Rome  Air  Development  Center  (RADC)  sponsored  a 
survey  by  Hughes  Aircraft  Company  in  an  effort  to  gain 
insight  into  the  factors  that  affect  the  field  reliability 
of  avionics  equipment.  In  an  analysis  of  sixteen  differ¬ 
ent  avionics  systems,  Hughes  reached  several  conclusions 
which  emphasized  the  need  for  clear  contract  term  defini¬ 
tion,  close  contractor/USAF  interaction,  and  realistic 
contractor  reliability  estimates  (5:56). 

The  study  determined  that  the  reliability  experi¬ 
enced  was  found  to  be  influenced  by  the  maintenance  per¬ 
sonnel  skill  levels,  maintenance  procedures,  and  geo¬ 
graphic/climatic  differences  of  geographically  separated 
bases . 

Contractually  negotiated  definitions  of  system 
failure  were  found  to  conflict  with  the  Air  Force  main¬ 
tenance  data  management  system;  the  fact  that  a  signifi¬ 
cant  portion  (44  percent)  of  reported  field  maintenance 
actions  did  not  involve  relevant  equipment  failures  empha¬ 
sized  the  disparity  (5:56).  An  analysis  of  the  contractor 
predicted  reliability  of  the  equipment  compared  to  a 
reassessed  relevant  failure  rate  indicated  that  some  con¬ 
tractor  estimates  were  optimistic.  To  underscore  the  impor¬ 
tance  of  properly  trained  and  motivated  avionics  maintenance 
personnel,  the  Hughes  study  noted  that 

.  .  .  fairly  large  differences  in  field  MTBF  can 
and  do  occur  even  though  the  same  equipment  is  used  on 
the  same  aircraft  flying  a  virtually  identical  training 
syllabus  at  different  bases  [5 :5a1]. 
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CHAPTER  III 


METHODOLOGY 

General  Approach  to  the  Problem 
In  order  to  place  the  field  reliability  and  main¬ 
tainability  performance  of  the  APG-66  in  perspective,  it 
was  necessary  to  research  those  events  leading  up  to  the  ; 

radar's  production  that  signif icantly  affected  the  equip¬ 
ment  development.  As  such,  those  planning  and  testing 
documents  relating  to  APG-66  development  milestones  were 
considered  the  logical  point  of  departure  for  further  study. 

Among  these  were  the  critical  design  review  of  the  radar, 
and  the  development  progress  demonstrated  in  the  reliabil¬ 
ity  qualification  test  and  the  intermediate  level  maintain¬ 
ability  demonstration. 

The  APG-66  demonstrated  field  reliability  and  main¬ 
tainability  experience  constituted  the  principal  portion 
of  the  analysis.  AFM  66-1  reported  field  reliability  and 
maintainability  results  were  used  as  the  supporting  data 

i 

in  the  calculation  of  a  variety  of  performance  indicator  ■ 

parameters.  These  parameters  were  used  to  judge  system  » 

effectiveness  and  also  provided  a  basis  of  comparison  with 
an  earlier  generation  radar,  the  APQ-120. 
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Management  Documents 


This  analysis  of  the  APG-66  reliability  and  main¬ 
tainability  history  begins  with  an  examination  and  summary 
of  those  program  management  documents  considered  crucial 
to  the  development  of  the  production  radar. 

The  Critical  Design  Review  (CDR)  of  the  F-16  Fire 
Control  Radar,  Volumes  I  through  III,  presented  that  infor¬ 
mation  necessary  to  ascertain  the  emphasis  that  program 
management  placed  on  the  reliability  and  maintainability 
aspects  of  the  radar  development.  The  review  summarized 
the  subcontractor's  conduct  of  the  entire  acquisition, 
including  system  design,  parts  control,  system  performance 
predictions,  test  programs  and  contractual  compliance.  The 
summary  is  an  important  foundation  for  future  analyses  since 
it  provides  a  baseline  for  comparison  of  operational 
results. 

The  final  report  of  the  conduct  and  results  of  the 
formal  Reliability  Qualification  Test  (RQT)  demonstrated 
the  radar's  ability  to  meet  contractual  MTBF  requirements 
and  also  provided  a  baseline  of  comparison  for  APG-66  opera¬ 
tional  reliability  performance.  In  this  analysis,  the 
emphasis  is  placed  on  the  test  results  in  comparison  with 
the  critical  design  review  predictions. 

It  was  considered  essential  at  the  RQT  point  to 
begin  comparison  with  the  APQ-120  in  terms  of  qualification 
test  programs,  measureable  reliability  performance,  and 
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failure  distributions.  This  analysis  provides  a  baseline 
for  further  operational  comparisons  of  the  two  radars  in 
Chapter  V. 

The  final  analysis  of  pre-production  radar  testing 
and  development  is  performed  in  the  scrutiny  of  the  con¬ 
tractual  requirements  and  test  results  of  the  formal  Inter¬ 
mediate  Level  Maintainability  Demonstration.  This  report 
provides  a  foundation  of  final  pre-production  testing 
results  to  be  used  in  the  analysis  of  operational  APG-66 
maintainability.  A  significant  amount  of  emphasis  is 
placed  on  the  conduct  of  the  demonstration  as  it  relates 
to  operational  maintenance  procedures.  The  maintainability 
demonstration  results  are  used  in  Chapter  V  in  comparison 
with  APG-66  maintainability  experience  to  provide  insight 
into  the  applicability  of  the  radar  maintainability  program 
and  the  demonstration  itself. 

Comparison  of  Field  Data 

Data  Sources 

In  order  to  properly  analyze  and  compare  the  APG-66 
field  reliability  performance,  it  was  necessary  to  obtain 
as  much  of  the  system's  reliability  and  maintainability 
history  as  possible.  The  necessary  data,  in  the  form  of 
RCS  LOG-MMO(AR)  7170  maintenance  action  summaries,  was 
obtained  for  a  twenty-month  operational  period  from  the 
Reliability  Management  section  of  the  F-16  Systems  Program 
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Office  (SPO) ,  located  at  Aeronautical  Systems  Division, 
Wright-Patterson  AFB.  This  report  provides  on-  and  off- 
equipment  historical  information  on  the  maintenance  actions 
man-hours,  and  aborts  for  a  six-month  period  by  month  on 
every  work  unit  code  included  in  the  master  record.  In 
addition,  the  report  provides  a  series  of  summary  line 
entries  for  each  subsystem  of  the  APG-66.  Since  the  pri¬ 
mary  use  of  the  report  is  for  reliability  and  maintainabil¬ 
ity  studies,  the  data  provides  the  capability  to  plot  trend 
and  performance  analyses  in  the  areas  of  system  and  com¬ 
ponent  failures,  maintenance  actions,  manpower  resource 
expenditure  and  aborts  (1:1). 

In  conjunction  with  RCS  LOG-MMO (AR)  7170,  RCS  LOG- 
MMO(AR)  7179  titled  "Summarized  Maintenance  Actions  for 
Selected  Work  Unit  Codes,"  was  used  to  analyze  and  compare 
the  maintainability  performance  of  the  APG-66.  The  report, 
obtained  from  Headquarters  Air  Force  Logistics  Command 
(AFLC)  located  at  Wright-Patterson  AFB,  provides  six  months 
of  summarized  detail  maintenance  Information  segregated  by 
on-equipment  and  shop  repair  actions.  The  report  is  pri¬ 
marily  used  to  conduct  detailed  analysis  on  work  unit  codes 
to  determine  reasons  for  substandard  performance,  to  deter¬ 
mine  if  maintenance  actions  are  evenly  distributed  geo¬ 
graphically,  and  to  identify  causes  of  high  unscheduled 
maintenance  rates  (2:2). 
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The  final  source  of  data,  used  in  comparison  with 
APG-66  reliability  development  and  field  experience,  was 
a  1973  radar  reliability  study  titled  "Research  Study  of 
Radar  Reliability  and  Its  Impact  on  Life-Cycle  Costs  for 
the  APQ-113 , -114 , -120  and  -144  Radars."  The  document, 
available  from  the  Defense  Technical  Information  Center,  is 
a  comprehensive  comparison  of  the  radars'  reliability 
activities  and  their  impacts  on  life-cycle  costs.  This 
thesis  effort  uses  some  of  the  methodologies  and  compara¬ 
tive  analyses  employed  in  that  study  as  a  basis  of  compari¬ 
son  for  the  APG-66. 

Field  Analyses 

The  analysis  of  APG-66  field  reliability  begins 
with  a  comparison  of  experienced  field  reliability  with 
those  figures  predicted  during  the  critical  design  review 
and  reliability  qualification  test.  In  order  to  shed  light 
on  the  radar's  predicted  reliability  performance,  the 
derivation  of  these  numbers  from  the  critical  design  review 
reliability  growth  model  is  presented.  Comparisons  of 
field  reliability  performance,  unadjusted  and  adjusted  for 
MTBF  definitional  variability,  with  those  pre-production 
radar  predictions  concludes  this  analysis. 

The  second  analysis  calculates  the  classic  measure- 
able  reliability  parameters  exhibited  in  the  APG-66 's  rela¬ 
tively  short  operational  deployment.  An  analysis  of  the 
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radar's  failure  rate  (X)  characteristics  is  performed  in 
order  to  determine  whether  the  system  has  achieved  those 
properties  identified  by  components  in  useful  life. 

The  reliability  growth  of  the  APG-66  discussed 
earlier  is  further  analyzed  in  this  section  using  reported 
field  data.  The  final  analysis  in  this  discussion 
researches  the  effect  of  increased  system  use  versus  the 
system's  MTBF. 

The  analysis  of  field  repair  times  begins  the 
examination  of  APG-66  maintainability.  The  section  spe¬ 
cifically  establishes  those  classical  parameters  that  are 
characteristic  of  and  indicative  of  equipment  maintain¬ 
ability.  The  focus  of  the  analysis  in  this  discussion  is 
in  determining  the  existence  of  a  learning  curve  effect  in 
the  maintainability  of  the  APG-66.  The  analysis  concludes 
with  an  examination  of  the  effect  on  maintainability  as 
increasing  numbers  of  radars  are  introduced  into  opera¬ 
tional  use. 

The  next  section  employs  data  gained  from  RCS 
LOG-MMO (AR)  7179  in  order  to  determine  the  field  flight¬ 
line  fault  diagnostic  capability  of  the  APG-66  maintenance 
concept.  The  APG-66  maintainability  section  is  concluded 
with  a  comparison  of  predicted  versus  field  line- 
replaceable  unit  (LRU)  mean-time-to-repair  (MTTR)  figures. 

Chapter  V  is  concluded  with  a  comparison  of  the 
APG-66  and  APQ-120.  These  radars'  capabilities  and 
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qualification  test  programs  are  presented  in  order  to 
establish  a  baseline  for  further  system  comparison.  This 
section  uses  the  field  reliability  and  maintainability 
figures  calculated  earlier  in  this  thesis  effort  to  judge 
the  relative  worth  of  the  reliability  and  maintainability 
activities  employed  during  the  APG-66  development. 
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CHAPTER  IV 


APG-66  RELIABILITY /MAINTAINABILITY  DEVELOPMENT 

Reliability  and  Maintainability  Overview 
The  focus  of  this  chapter  is  an  examination  of  the 
reliability  and  maintainability  historical  development  of 
the  APG-66  fire  control  radar.  The  analysis  begins  with 
the  examination  of  the  state  of  APG-66  design  during  the 
critical  design  review  where  the  reliability  and  maintain¬ 
ability  performance  figures  are  used  as  a  baseline  for 
future  system  development.  Formal  reliability  qualifica¬ 
tion  testing  (RQT) ,  performed  to  demonstrate  final  APG-66 
design  MTBF ,  is  the  second  subject  of  analysis.  The  empha¬ 
sis  in  this  section  is  placed  on  the  achieved  testing 
results  in  comparison  with  the  F-4  APQ-120  at  a  similar 
stage  of  development.  The  final  analysis  examines  the 
requirements  and  demonstrated  results  of  the  APG-66  formal 
intermediate  level  maintainability  demonstration. 

In  summary,  the  underlying  objective  of  this  chapter 
is  to  gain  insight  into  the  progression  in  the  reliability 
and  maintainability  design  of  the  APG-66  fire  control 
radar.  This  information,  in  combination  with  the  opera¬ 
tional  field  performance  data  analysis  in  Chapter  V,  will 
be  used  to  assess  the  overall  system  reliability  and  main¬ 
tainability  program. 
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Pre-production  Reliability  and 
Maintainability  Development 

Critical  Design  Review 

Reliability  Emphasis  and  Predictions.  The  critical 
design  review  (CDR)  of  the  APG-66,  conducted  in  June  1976, 
placed  a  great  deal  of  emphasis  on  the  reliability  perform¬ 
ance  of  the  system.  Four  measures  developed  to  assure 
APG-66  reliability  were  evident  in  the  reliability  program 
proposed  during  the  review.  The  first  measure  considered 
was  the  inherent  strengths  of  the  system's  full-scale 
development  design  which  had  been  subjected  to  electrical/ 
thermal  derating,  design  and  thermal  analysis,  failure 
mode  effects  analysis  and  predicted  reliability  analysis 
(20:p.3-4).  The  second  measure  involved  selection  of  only 
high  quality /reliability  component  parts  for  inclusion  in 
the  system.  General  Dynamics,  as  the  prime  contractor, 
required  that  major  vendors  use  preferred  parts,  source 
control  nonstandard  parts,  burn-in  and  temperature-cycle 
final  assemblies  and  improve  upon  the  radar  design  to  meet 
the  contracted  MTBF  (20:p.3-24).  The  third  measure,  the 
test-and-fix  phase,  involved  environmental  screening  and 
thermal  verification,  test  and  evaluation,  and  reliability 
growth  measures.  Finally,  Westinghouse  as  the  APG-66  sub¬ 
contractor,  was  required  to  demonstrate  both  in  formal 
testing  and  operational  use  the  contractual  MTBFs.  As  a 
full-scale  development  design,  the  system  was  required  to 


29 


meet  a  formal  reliability  qualification  test  (discussed 
later  in  this  chapter)  MTBF  of  sixty  hours  for  the  entire 
radar  and  seventy  hours  for  the  radar  less  the  ground¬ 
mapping  elements  (20:p.3-4).  Once  full-scale  production 

’  o 

began,  the  radar  was  required  to  exhibit  100/125  hours  in 
production  acceptance  testing  and  maintain  a  warrantied 
guaranteed  mean-time-between-failure  of  seventy  hours  dur¬ 
ing  operational  use  (20:p.3-4). 

The  F-16  radar  reliability  test  program  was  fore¬ 
cast  to  be  very  comprehensive  in  that  it  included  various 
environmental  screenings  in  addition  to  three  formal  reli¬ 
ability  demonstrations.  Table  1  depicts  the  test  program 
matrix  annotated  with  the  originating  requirement,  sample 
sizes  and  appropriate  test  comments.  It  should  be  noted 
that  a  100  percent  sample  of  all  pre-production  and  produc¬ 
tion  radar  comp  'ent  modules  (SRUs)  and  first-line  units 
(LRUs)  are  exposed  to  burn-in  and  random  vibration. 

The  inherent  reliability  predictions  calculated 
during  the  critical  design  review  were  very  optimistic 
about  projected  field  performance.  Table  2  presents  the 
inherent  reliability  predictions  for  the  air-to-air  and 
combined  air-to-air  and  air-to-ground  modes  during  the 
full-scale  development  and  production  phases.  Table  2 
will  be  used  in  Chapter  V  to  compare  predicted  reliability 
with  operational  results  in  order  to  gain  further  insight 
into  the  reliability  program. 
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TABLE  1 


F-16  RADAR  RELIABILITY  TEST  PROGRAM 


Test 

Cement 

General 

Dynamics 

VfeSt- 

inghouse 

Full 

Scale 

Develop¬ 

ment 

Pro¬ 

duction 

Sample 

Module 

Burn-In 

Temperature 

Shock 

X 

X 

X 

ALL 

First  Line 
Uhit  (FLU) 
Bum-In 

Temperature 

Randan 

Vibration 

X 

X 

X 

ALL 

Reliabil¬ 
ity  Quali¬ 
fication 
Test 

MH/-STD-781B, 
Test  Plan  III, 
Modified  Test 
Level  F 

X 

X 

X 

3,5 

Thermal 
Verifica¬ 
tion  Test 

TVio  Tempera¬ 
ture  Bakes 

X 

X 

1 

Reliabil¬ 
ity  Accept¬ 
ance  Test 

MEL-STO-781B , 
Test  Plan  I\A 
Modified  Test 
Level  F 

X 

X 

8 

Transmitter  MIL-STD-781B,  X 

Waveguide-  Modified  Test 

RQT  at  Level  F,  3000 

Vendor  HR,  One  Failure 

Permitted 

X 

3 

Reliabil¬ 
ity  Growth 

Temperature 
Cycle,  Random 
Vibration, 
Humidity-2, 000 
Hours 

X 

X 

X 

2 

SOURCE:  Critical  Design  Review  F-16  Fire  Control 
Radar,  p.  3-5. 
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TABLE  2 


INHERENT  RELIABILITY  PREDICTIONS 


LRU 

Full-Scale 

Development 

Production 

Combined 

Modes 

Air-to- 

Air 

Combined 

Modes 

Air-to- 

Air 

MTBF 

(hours) 

MTBF 

(hours) 

MTBF 

(hours) 

MTBF 

(hours) 

Antenna 

1,764 

1,764 

1,764 

1,764 

LPRF 

651 

651 

651 

651 

Transmitter 

1,366 

1,366 

1,366 

1,366 

DSP 

491 

548 

576 

693 

Computer 

1,259 

1,259 

2,008 

2,008 

RCP 

12,048 

12,048 

12,048 

12,048 

System 

168 

174 

187 

197 

SOURCE:  Critical  Design  Review  (U)  F-16  Fire  Con¬ 
trol  Radar,  p.  3-7. 
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Maintainability  Emphasis  and  Predictions.  Westing- 


house  placed  a  great  deal  of  emphasis  on  the  maintainabil¬ 
ity  features  of  the  APG-66  system  prior  to  and  during  the 
critical  design  review  of  June  1976.  The  maintainability 
program  milestones  developed  by  Westinghouse  proved  to  be 
somewhat  optimistic,  however,  as  is  discussed  later  in  this 
chapter.  The  original  milestone  plan,  as  presented  at  the 
critical  design  review,  is  reflected  in  Figure  1. 

The  culmination  of  the  maintainability  program,  the 
intermediate  maintainability  demonstration,  was  forecast 
to  have  two  problem  areas,  one  of  scheduling  Avionics 
Intermediate  Shop  (AIS)  test  equipment  that  might  delay 
the  demonstration  itself.  The  second  problem  was  a  matter 
of  normalizing  diverse  maintenance  repair-time  data  to  a 
common  base  for  contractual  interpretation  of  mean-time-to- 
repair  (MTTR) .  The  latter  was  a  result  of  performing  APG-66 
fault  diagnosis  during  the  test  by  different  methods: 
manually,  using  factor  test  equipment  and  automatically, 
using  AIS  equipment  (20:p.l0-4). 

The  basic  parameter  to  measure  system  maintain¬ 
ability,  mean-time-to-repair ,  and  its  prediction  was  the 
foundation  of  the  APG-66  maintainability  program.  The  sub¬ 
contractor  utilized  the  guidelines  of  MIL-HDBK-472  in 
generating  a  computer  program  that  could  provide  MTTR  pre¬ 
dictions  on  a  periodic  and  timely  (bi-monthly)  basis.  The 
basic  data  used  in  the  MTTR  predictions  came  from  four 
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SOURCE:  Critical  Design  Review  (U)  F-16  Fire  Control  Radar,  p.  10- 


sources  (20 :p. 10-8):  (a)  the  inherent  failure  rates  of  the 
SRUs,  (b)  the  test  times  achieved  in  isolating  faults  to 
specific  SRUs,  (c)  the  remove  and  replace  times  for  the 
SRUs,  and  (d)  the  retest  times  for  a  repaired  LRU.  The  pre 
dieted  MTTR,  to  be  compared  with  the  contractual  require¬ 
ments,  was  computed  in  the  following  manner  (20:p.l0-8): 


where , 

\  =  failure  rate  (number  of  failures  per  10®  hours) 
and 

Mct  *  corrective  maintenance  time  to  repair  an  LRU. 


The  MTTR  predictions  shown  in  Table  3  were  calculated  in 
May  1976  and  are  presented  as  a  basis  for  comparative 
analysis  in  Chapter  V. 

The  APG-66  was  designed  to  require  no  adjustment  or 
calibration  of  the  SRUs  once  the  system  was  installed  in 
an  operational  aircraft.  The  critical  design  review  noted, 
however,  that  the  radar  transmitter  would  require  adjust¬ 
ments  in  two  instances,  a  not  unfavorable  situation  that 
still  related  the  subcontractor's  desire  to  conform  to  the 
maintainability  requirements  (20 :p. 10-14) . 

Inherent  in  the  maintainability  problem  was  the 
complexity  of  the  APG-66.  In  order  to  enable  a  high  per¬ 
centage  of  fault  isolation  and  to  reduce  components  to  a 
lesser  number  of  functional  modules,  the  six  LRUs  of  the 
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TABLE  3 


INTERMEDIATE  LEVEL  MTTR  PREDICTIONS 


LRU 

MTTR 

(Minutes) 

Maximum  Corrective 
Time  (Minutes) 

Antenna 

20.44 

23.90 

LPRF 

42.64 

49.14 

Transmitter 

86.31 

182.30 

DSP 

30.64 

34.70 

Computer 

30.21 

37.35 

RCP 

28.26 

44.41 

System 

42.05 

67.15 

SOURCE:  Critical  Design  Review  (U)  F-16  Fire  Con¬ 
trol  Radar,  p.  10-10. 

fire  control  radar  were  designed  as  eighty-seven  shop- 
replaceable  units  (SRUs).  During  the  critical  design  review 
this  maintenance  concept  was  instituted.  Analysis  revealed 
that  85  percent  of  detected  system  faults  could  be  isolated 
to  a  single  component  during  depot  level  maintenance.  In 
addition,  at  the  same  maintenance  level,  a  system  fault 
could  be  isolated  to  three  components  99  percent  of  the 
time  (20 :p. 10-15) . 

Fault  isolation  at  the  field  level  of  maintenance 
was  also  considered  important.  As  was  discussed  in 
Chapter  I,  the  self-test  and  built-in-test  features  of  the 
APG-66  were  important  facets  in  the  overall  maintenance 
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concept  for  fault  isolation.  At  the  time  of  the  critical 
design  review,  these  two  subsystems  were  exhibiting  impres¬ 
sive  accuracy  in  the  fault  isolation  process;  in  fact, 
each  had  in  excess  of  98  percent  of  correct  system  diag¬ 
nosis  (20 :p. 10-88) . 

In  summary,  the  APG-66  maintenance  concept,  main¬ 
tainability  design  and  maintainability  predictions  pre¬ 
sented  during  the  critical  design  review  were  very  optimis¬ 
tic  of  projected  field  results.  Later  sections  of  this 
chapter  present  further  analysis  and  test  results  during 
the  APG-66  maintainability  development. 

Reliability  Qualification  Test 

Introduction.  Westinghouse  Electric  Corporation, 
in  conjunction  with  General  Dynamics,  conducted  a  reliabil¬ 
ity  qualification  test  of  the  full-scale  development  F-16 
fire  control  radar  during  the  period  15  April  through 
30  June  1978.  The  objective  of  the  reliability  qualifica¬ 
tion  test,  which  was  performed  in  accordance  with  General 
Dynamics  Specification  16ZE009  and  MIL-STD-781B  Test 
Plan  III,  was  to  verify  the  hypothesis  that  the  true  MTBF 
of  the  FSD  F-16  fire  control  radar  configuration  met  the 
minimum  acceptable  contractual  MTBF  (21:p.l-l).  An  addi¬ 
tional  requirement  during  the  reliability  qualification 
test  was  that  the  fire  control  radar  pass  the  contrac¬ 
tually  mandated  sixty-hour  MTBF  for  the  entire  radar  and 
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the  seventy-hour  MTBF  requirement  for  the  radar  less  the 
elements  unique  to  the  ground  map  modes  (21:p.l-l).  The 
APG-66  was  subjected  to  two  reliability  qualification 
tests,  failing  the  first  after  thirty-three  hours  of  rele¬ 
vant  operating  time,  and  passing  the  second  in  which  a  total 
of  740  relevant  operating  hours  were  accumulated  (21:p.2-3). 

Results  of  APG-66  Reliability  Qualification  Testing. 
The  initial  reliability  qualification  testing  begun  on  the 
two  FSD  APG-66  radars  resulted  in  a  test  reject  decision 
when  three  relevant  failures  were  incurred  after  thirty- 
three  hours  of  relevant  testing  (21:p.2-l).  In  addition, 
twenty-two  discrepancies  were  discovered  during  pretest 
setup  and  nonrelevant  troubleshooting  time.  In  order  to 
prevent  recurrence  of  the  noted  deficiencies  in  subsequent 
formal  testing  and  operational  use,  proposed  corrective 
actions  were  submitted  to  the  prime  contractor  and  there¬ 
after  incorporated  into  the  radar  design.  Of  the  twenty- 
five  total  deficiencies,  thirteen  were  corrected  by  design 
specification  changes,  five  were  corrected  at  component 
vendors'  facilities,  four  were  workmanship  problems  that 
precipitated  procedural  changes,  and  no  cause  could  be 
identified  for  three  (21:p  2-2).  The  measures  implemented 
to  correct  the  pretest  deficiencies  were  thought  to  have 
accelerated  the  radar  environmental  design  maturity  such 
that  the  contractor  exhibited  a  high  level  of  confidence 
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in  immediately  resuming  formal  reliability  qualification 
testing. 

As  a  result,  the  second  formal  reliability  qualifi¬ 
cation  test  was  conducted  between  15  April  and  30  June  1978 
during  which  time  the  two  systems  achieved  721  relevant 
hours  of  operation  while  incurring  thirteen  relevant 
failures  (21:p.2-3).  The  point  estimate  of  fifty-seven 
hours  used  as  the  final  qualification  test  achieved  MTBF 
was  derived  by  extending  the  test  past  the  721-hour  test- 
plan  truncation  point  to  740  hours  (21:p.2-3).  Figure  2 
depicts  the  plot  made  of  the  progress  of  the  test  in  time 
based  on  the  accept/reject  curves  of  MIL-STD-781B  test 
plan  III.  Each  point  on  the  step  curve  represents  one 
failure  occurrence  in  either  of  the  two  radars  tested, 
plotted  against  the  combined  test  time  of  both  radars  at 
the  time  of  failure  (21:p.2-10).  The  graph  clearly  depicts 
the  sixty-hour  MTBF  acceptance  decision  after  the  eleventh 
failure  and  the  seventy-hour  acceptance  decision  after  the 
thirteenth  failure. 

Table  4  portrays  the  thirteen  relevant  failures 
observed  during  the  second  qualification  test,  each  allo¬ 
cated  to  the  particular  LRU  in  which  the  deficiency  was 
detected.  In  analysis,  when  the  observed  MTBF  for  each 
specific  LRU  is  compared  with  the  predicted  MTBF,  it  becomes 
obvious  that  the  low-power  radio  frequency  (LPRF)  and  trans¬ 
mitter  (XMTR)  LRUs  performed  well  below  what  was  expected. 
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TOTAL  TEST  TIME  (HOURS) 

Fig.  2.  Reliability  Qualification  Test 

SOURCE:  Final  Report  on  the  Reliability  Qualification  Test  of  the  F-16 
Fire  Control  Radar  (FSD) ,  p.  2-11. 


TABLE  4 


RELEVANT  FAILURES,  RELIABILITY 
QUALIFICATION  TEST 


LRU 

Failures 

Observed  MTBF 
(Hours) 

Predicted  MTBF 
(Hours) 

Antenna 

2 

370 

360 

LPRF 

5 

148 

360 

XMTR 

4 

185 

300 

DSP 

0 

Indeterminate 

800 

Computer 

1 

738 

480 

RCP 

1 

738 

480 

System 

13 

57 

70 

SOURCE:  Final  Report  on  the  Reliability  Qualifica¬ 
tion  Test  of  the  F-16  Fire  Control  Radar,  p.  2-4. 

The  figures  in  Table  4  will  also  be  used  in 


Chapter  V  as  a  basis  for  comparison  of  reliability  qualifi¬ 
cation  test  performance  versus  field  experience. 

In  order  to  correct  the  deficiencies  identified 
by  the  thirteen  relevant  failures,  the  subcontractor  pro¬ 
posed  design  and  procedural  changes  along  with  engineering 
investigation  to  General  Dynamics  to  be  implemented  in  the 
production  fire  control  radars.  The  contractor  predicted 
that  these  improvements,  gained  from  qualification  test 
experience,  would  significantly  improve  the  overall  system 
reliability. 
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Reliability  Qualification  Comparison  with  APQ-120. 


Since  reliability  qualification  testing  is  the  formal 
measure  of  achieved  system  MTBF,  the  APQ-120  RQT  No.  5  and 
the  APG-66  RQT  No.  2  tests  were  compared  as  shown  in 
Table  5.  The  complexity  (parts  count)  of  the  systems  were 
similar,  APQ-120  and  13,500  versus  APG-66  with  15,100; 
however,  the  APG-66  contract  MTBF  was  nearly  seven  times 
more  demanding  than  that  for  the  APQ-120  (sixty  hours 
versus  nine  hours) .  In  addition,  the  APG-66  demonstrated 
95  percent  of  its  contractual  MTBF,  while  the  APQ-120  fell 
short  of  its  contractual  requirement  by  more  than  50  per¬ 
cent.  The  overall  effect  of  the  advancements  in  component 
technology,  system  design  and  equipment  reliability  programs 
is  nowhere  more  revealing  than  in  the  measureable  relia¬ 
bility  performance  (MTBF)  differential  where  the  ratio  is 
13:1  in  the  favor  of  the  APG-66. 

While  only  two  APG-66  radars  were  evaluated  during 
the  test,  the  fact  that  each  APG-66  unit  achieved  signifi¬ 
cantly  more  test  exposure  hours  per  unit  of  equipment 
(369  hours  versus  32  hours)  results  in  a  greater  chance 
of  detecting  workmanship-related  defects  whose  manifesta¬ 
tion  is  environmentally  or  time  dependent  (7:168).  Never¬ 
theless,  the  opportunity  to  subject  a  greater  sample  to 
formal  reliability  qualification  testing  should  be  enforced 
in  order  to  determine  if  subtle  defects  in  component  parts 
lots  exist. 
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TABLE  5 


APG-66  VERSUS  APQ-120  RELIABILITY 
QUALIFICATION  TEST  COMPARISON 


APQ-1203 

APG-66 

Requirements 

Contract  MEBF 

9  hours 

60/70  hours 

Results 

Demonstrated  MTBF 

4.3  hours 

57  hours 

Test  Conditions 

Chamber  Test  Levels 

-54°C  to 

-54 °C  to 

49°C 

71  °C 

Vibration 

2.2  g 

2.96  g 

Relevant  Hours 

96 

738 

Resources 

Number  of  Equipments 

3 

2 

Test  Duration 

Elapsed  Months 

3 

2.5 

Efficiency 

Average  Relevant 
Hours/Month 

32 

295 

Incidents 

Part  Design 

0 

1 

Part  Workmanship 

43 

5 

Design 

0 

2 

Workmanship 

25 

4 

undetermined 

7 

11  (10  non- 

relevant) 

Total  Incidents 

75 

23 

Incident  Cate- 

Part 

57 

26 

gories  by 

Percent  of 

Design 

0 

9 

Total  Incidents 

Workmanship 

33 

17 

undetermined 

10 

52  (48  non- 

relevant) 

Incident  Classi¬ 
fication 

Relevant  Failures 

16 

13 

SOURCE:  Research  Study  of  Radar  Reliability  and  Its 
Impact  on  Life-Cycle  Costs  for  the  APQ-113 ,-114 ,-120,  and 
-144  Radars,  p.  169. 
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The  reliability  qualification  test  failure  distri¬ 
butions  for  both  systems  were  segregated  into  three  groups 
according  to  the  determination  of  the  cause  of  the  failure — 
parts  (design  and  workmanship) ,  equipment  design  and  equip¬ 
ment  workmanship.  Table  5  depicts  these  failure  classifi¬ 
cations  and  the  associated  percentage  of  failures.  Several 
items  of  interest  are  apparent  from  the  data;  most  note¬ 
worthy  is  the  fact  that  only  4  percent  of  the  APG-66  rele¬ 
vant  failures  had  an  undetermined  cause.  It  can  be  assumed 
that  the  test  equipment  employed  did  in  fact  perform  its 
function  to  a  high  degree.  Another  significant  revelation 
is  the  fact  that  parts  workmanship  and  equipment  workman¬ 
ship  incidents  accounted  for  70  percent  of  the  relevant 
failures.  Taking  into  consideration  the  low  absolute 
number  of  relevant  failures,  this  may  be  attributed  to  poor 
component  vendor  parts  control  and  a  lapse  in  the  LRU 
environmental  screening  program..  Further  examination  of 
the  table  reveals  that  the  APG-66  system  design  appears  to 
be  quite  mature  in  its  early  stage  of  development  in  that 
only  two  relevant  failures  could  be  assigned  an  equipment 
design  cause. 

Table  6  portrays  a  comparison  of  the  two  fire  con¬ 
trol  radars'  qualification  test  failures  distributed  accord¬ 
ing  to  the  system  environment  when  the  deficiency  was  dis¬ 
covered.  In  the  case  of  the  APQ-120,  more  failures  were 
observed  in  the  cold  environment  than  in  the  hot  by  a 
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factor  of  3:1.  In  determining  the  cause  for  each  relevant 
failure,  however,  the  final  report  of  APG-66  RQT  ascer¬ 
tained  that  only  one  failure  was  due  to  the  test  thermal 
(hot)  environment.  This  demonstrates  the  importance  and 
effectiveness  of  a  thorough  system  environmental  screening 
program. 

One  significant  fact  is  that  five  of  the  total 
relevant  APG-66  failures  were  found  to  be  caused  by  the 
vibration  employed  during  the  testing.  This  can  be  attri¬ 
buted  to  the  extensive  employment  of  this  environment 
(2.96  g's  for  fourteen  minutes  of  every  hour  of  system 
operation) . 

One  important  result  of  reliability  qualification 
testing  is  the  fact  that  better  quality  and  more  reliable 
equipment  will  be  produced  when  the  test  cycle  used  closely 
simulates  field  conditions.  The  test-and-fix  atmosphere 
of  the  formal  testing  ensures  a  major  contribution  will  be 
made  to  the  future  reliability  growth  of  the  equipment. 

Intermediate  Level  Maintain¬ 
ability  Demonstration 

Contractual  Requirements.  As  part  of  the  effort  to 
deliver  a  fire  control  radar  that  was  field  maintainable. 
General  Dynamics  required  that  the  radar  subcontractor, 
Westinghouse ,  demonstrate  intermediate  level  maintenance 
capability  to  fault  isolate  the  radar  LRUs  to  a  shop 
replaceable  unit  (SRU) ,  within  specific  accessibility, 
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adjustment,  skill  level  and  mean-time-to-repair  (MTTR) 
parameters  (22:1).  The  tests  were  conducted  on  the  radar 
LRUs  that  are  the  focus  of  this  thesis  effort:  antenna, 
radar  computer,  radar  control  panel,  transmitter,  digital 
signal  processor  and  low-power  radio  frequency.  In  order 
that  the  maintainability  demonstration  duplicate  field 
conditions  as  nearly  as  possible,  the  test  attempted  to 
utilize  diagnostic  equipment  (Avionics  Intermediate  Shop) 
actually  deployed  with  operational  units;  however,  in  some 
instances  Westinghouse  factory  test  equipment  and  manual 
testing  were  used. 

The  intermediate  level  maintainability  demonstra¬ 
tion  consisted  of  inserting  150  faults  into  the  six  LRU 
shop  replaceable  units  (SRU)  and  computing  the  time  param¬ 
eters  required  by  General  Dynamics:  fault  isolation,  LRU 
retest,  and  remove /replace  and  alignment  (22:3).  The  fault 
isolation  parameter  was  the  time  required  to  isolate  a 
fault  in  a  defective  SRU  utilizing  factory  test  equipment  or 
AIS;  the  remove  and  replace  time  was  the  time  required  to 
remove  an  SRU  that  had  been  determined  to  be  defective; 
the  total  repair  time  was  the  sum  of  the  two  aforementioned 
parameters  plus  a  baseline  test  time,  a  measure  which  deter¬ 
mined  that  a  corrected  SRU  functioned  properly  (22:32). 
General  Dynamics  Prime  Item  Development  Specification 
16ZE009H  required  that  the  values  measured  during  the  tests 
be  used  to  determine  the  intermediate  level  maintainability 
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parameters  to  ascertain  progress  of  the  maintainability 
development  program.  These  parameters  and  requirements, 
in  accordance  with  the  specification,  were  as  depicted  in 
Table  7. 


TABLE  7 


INTERMEDIATE  LEVEL  MAINTAINABILITY  REQUIREMENTS 


Parameter 

Requirement 

1. 

Maintenance  Time 

1.0  hour 

Mean  (M  .  ) 
ct 

1 . 0  hour 

2 . 0  hours 

2. 

Skill  level 

AFM  39-1,  Level  5 

3. 

Intermediate  Level 
Testability 

97  percent 

4. 

Adjustments 

No  Adjustments  at 
Intermediate  Level 
(Except  as  approved) 

5. 

Accessibility 

Direct  Access  to  each 

SRU  (Except  as  approved) 

SOURCE:  Intermediate  Level  Maintainability  Assess¬ 
ment  and  Demonstration  Report  for  the  P-16  Fire  Control 
Radar ,  p .  5 . 


Intermediate  Level  Results.  The  computation  of  the 
final  results  of  the  intermediate  level  maintainability 
demonstration  was  based  on  the  actual  repair  times  recorded 
on  the  isolation  and  repair  of  138  faults.  The  calcula¬ 
tions  revealed,  with  no  normalization  adjustment  for  manual 
test  mode,  repair  time  results  that  were  outside  the 
General  Dynamics  specification  requirements  of: 
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Mean  (M  . )  1.0  hour 

ct 

Max  (Mmaxct)  2.0  hour 

In  order  to  present  these  parameters  in  terms  of  the  spe¬ 
cification  requirements  that  were  based  on  the  use  of 
automatic  test  equipment,  a  decision  was  made  to  modify 
the  recorded  data  for  the  low-power  radio  frequency,  trans¬ 
mitter,  and  antenna  test  times  which  were  diagnosed  with 
nonautomatic  equipment.  The  modified  final  results  of 

Mean  (M  )  =  j8 1  07" 

ct 

Max  ^maxct*  "  1  hour  48 '  13" 

demonstrated  the  APG-66  system's  capability  to  meet  the 

contractual  specifications  of  repair  time. 

The  requirements  of  APM  39-1,  Skill  Level  5  for 

Air  Force  Specialty  Code  (AFSC)  32550,  Fire  Control  Systems 

Mechanic,  and  AFSC  32551  were  demonstrated  by  inspection 

during  the  test  to  be  more  rigorous  than  the  requirements 

placed  on  the  operator  during  the  test  demonstration. 

The  unabridged  requirement  for  parameter  number 

three,  intermediate  level  testability ,  read  in  full: 

Each  LRU  shall  contain  sufficient  test  points  in 
external  SE  connectors,  internal  test  connectors 
(DSP)  only  and  normal  LRU  interface  connectors  to 
permit  connection  of  the  required  test  equipment  to 
allow  positive  fault  isolation  to  an  SRU  for  a  minimum 
of  97%  of  all  failures  attributable  to  the  system's 
SRU  total  failure  rate  [22:35]. 
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The  initial  calculations  of  fault  isolation  yielded  a  poor 
testability  of  68  percent;  however,  after  corrective  action 
and  retest  a  94  percent  testability  was  achieved  (22:36). 
The  radar  subcontractor  considered  this  diagnostic  figure 
to  be  sufficient  and  recommended  to  General  Dynamics  that 
the  97  percent  requirement  be  changed  to  94  percent 
(22:37) . 

The  demonstration  met  the  requirements  of  parameter 
number  four,  adjustments,  except  for  two  SRUs  that  were 
exempted  by  prior  approval  from  General  Dynamics.  The  last 
parameter,  SRU  accessibility,  was  verified  for  each  LRU 
by  demonstration  during  the  test  (22:38). 

In  summary,  maintainability  in  the  design  of  the 
radar  produced  satisfactory  results  during  the  intermediate 
level  maintainability  demonstration.  Chapter  V  will  com¬ 
pare  operational  repair  times  and  diagnostic  success  with 
the  outcomes  of  this  demonstration  in  order  to  provide 
insight  into  the  radar  maintainability  program. 
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CHAPTER  V 


FIELD  RELIABILITY  AND  MAINTAINABILITY 
DATA  ANALYSIS 

Introduction 

The  general  focus  of  this  chapter  is  an  examination 
of  the  field  reliability  and  maintainability  of  the  APG-66 
fire  control  radar.  The  reliability  analysis  begins  in 
the  next  section  with  a  comparison  of  reported  field  reli¬ 
ability  performance  data  as  opposed  to  the  contractual  equip¬ 
ment  reliability  requirements  of  the  last  chapter.  The 
second  analysis  determines  those  classical  reliability  mea¬ 
sures  exhibited  by  the  APG-66  in  its  deployed  phase. 

The  following  section  of  this  chapter  concentrates 
on  the  reported  field  maintainability  of  the  APG-66.  In 
addition  to  researching  the  indicators  of  field  maintain¬ 
ability,  the  section  compares  the  operational  maintainabil¬ 
ity  results  with  the  critical  design  review  and  demonstra¬ 
tion  test  predictions  in  order  to  determine  the  results  of 
the  dedicated  maintainability  program. 

The  final  section  integrates  the  reliability  and 
maintainability  discussion  by  comparing  the  APG-66  with  the 
performance  of  an  earlier  generation  radar,  the  APQ-120. 

In  this  comparison,  it  should  be  kept  in  mind  that 


51 


equipment  technical  capabilities  are  an  important  facet  of 
equipment  mission  performance  but  are  not  the  focus  of  this 
effort . 


APG-66  Field  Reliability 

Objective 

The  primary  objective  of  this  section  is  to  corre¬ 
late  differences  in  reported  field  reliability  performance 
data  for  the  APG-66  in  comparison  with  equipment  reliabil¬ 
ity  requirements.  A  second  objective  is  to  determine  the 
reliability  parameters  experienced  by  the  system  in  its 
deployed  phase. 

The  data  used  in  the  analysis  comes  from  several 
sources.  The  performance  characteristics  of  reliability 
qualification  testing  used  in  Tables  8,  9  and  Figure  3 
(shown  later  in  this  chapter  are  derived  from  the  "Final 
Report  on  the  Reliability  Qualification  Test  of  the  F-16 
Fire  Control  Radar  (FSD) , "  a  document  prepared  by  the 
Westinghouse  Defense  and  Electronic  Systems  Center. 
Reliability  predictions  for  the  production  radar  used  in 
Tables  8,  9  and  Figure  3  are  extracted  from  the  "Critical 
Design  Review  of  the  F-16  Fire  Control  Radar,"  a  document 
also  prepared  by  Westinghouse.  The  data  used  to  calculate 
the  radar’s  field  reliability  results  in  Tables  8  and  9 
were  obtained  for  a  six-month  reporting  period  (August  1980 
to  January  1981)  from  RCS  LOG-MMO (AR)  7170,  titled 
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"Maintenance  Actions,  Manhours,  and  Aborts  by  Work  Unit 
Code,"  an  output  provided  by  the  Air  Force  Logistics  Com¬ 
mand  management  information  system. 

Data  Analysis 

Requirements  and  Field  Results  Comparison.  This 
analysis  of  APG-66  field  reliability  performance  begins 
with  an  examination  of  the  reliability  growth  model 
(Figure  3)  which  was  presented  at  the  critical  design  review 
as  part  of  the  Westinghouse  reliability  management  program. 
The  reliability  growth  model  is  a  graphic  representation 
that  can  be  used  to  ascertain  the  amount  of  reliability 
testing  hours  necessary  to  achieve  a  target  MTBF .  Con¬ 
versely,  it  can  also  be  used  during  development  to  deter¬ 
mine  whether  a  system's  reliability  is  growing  at  the  pro¬ 
jected  rate. 

The  subcontractor's  selected  growth  rate  for  the 
APG-66  radar,  a=0.5,  was  based  on  the  radar's  inherent 
design,  the  component  inherent  failure  rates  and  the  sys¬ 
tem's  parts  complement.  An  examination  of  Figure  3  reveals 
that  in  order  to  achieve  a  target  sixty  hours  MTBF,  a 
minimum  of  1500  hours  of  evaluation  testing  would  be 
required  at  the  selected  growth  rate  of  a=0.5.  It  is  sig¬ 
nificant  that  although  the  reliability  qualification  test 
results  were  less  than  that  contractually  specified 
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PRODUCTION  BASELINE  PREDICTION 
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(fifty- seven  hours  MTBF  versus  sixty  hours  MTBF) ,  the 
radar  experienced  a  greater  reliability  growth  rate  (a=0.6) 
than  that  required. 

The  reliability  growth  model  is  also  used  to  estab¬ 
lish  the  testing  hours  necessary  to  achieve  the  production 
baseline  and  production  acceptance  test  requirements.  The 
test  consists  of  a  2000-hour  test-and-fix  development  con¬ 
ducted  in  accordance  with  test  plan  IV-A  of  MIL-STD-781B, 
where  a  minimum  MTBF  of  100  hours  at  a  growth  rate  of 
a=0.5  is  required.  Although  the  production  acceptance 
tests  are  not  yet  complete,  the  minimum  and  baseline  produc¬ 
tion  reliability  figures  of  100  and  232  hours  MTBF,  respec¬ 
tively,  constitute  a  basis  for  comparison  with  the  field 
reliability  performance  data  experienced  to  date. 

Table  8  provides  reliability  figures  that  can  be 
used  to  judge  the  accuracy  of  the  program's  reliability 
design  and  testing  predictions  in  comparison  with  the  field 
results.  The  figures  in  column  one  represent  the  subcon¬ 
tractor's  predicted  MTBFs,  which  were  based  primarily  on 
the  radar's  inherent  design,  presented  during  the  critical 
design  review  of  the  radar.  The  data  in  column  2,  RQT 
Results,  are  reliability  performance  figures  obtained 
during  formal  reliability  qualification  testing.  Column  3 
presents  the  reliability  experienced  by  the  radar  during  a 
six-month  operational  period. 
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TABLE  8 


COMPARISON  OF  PREDICTED  RELIABILITY  VERSUS 
OPERATIONAL  RESULTS,  APG-663 


LRU 

Production*3 

Prediction 

RQTC 

Results 

Fieldd 

Results 

Antenna 

1764 

370 

138 

LPRF 

651 

148 

150 

Transmitter 

1366 

185 

202 

DSP 

576 

I  nde  te  rminate 

399 

Computer 

2008 

738 

300 

System 

187 

57 

35 

aHours  MTBF. 

^SOURCE :  Critical  Design  Review  (U)  F-16  Fire  Con¬ 
trol  Radar,  p.  3-7. 

c SOURCE :  Final  report  on  the  Reliability  Qualifica¬ 
tion  Test  of  the  F-16  Fire  Control  Radar  (FSD) ,  p.  2-4. 

Reporting  Period:  August  1980  to  January  1981. 

The  significant  observation  of  Table  8  is  the  dif¬ 
ference  between  the  system  MTBF  in  the  RQT  results  (57)  and 
that  experienced  in  the  field  (35).  This  can  be  explained 
by  the  different  definitions  of  MTBF  employed  in  the  two 
phases.  In  the  reliability  qualif ication  testing  phase, 
MTBF  was  defined  as:  ".  .  .  equal  to  the  total  operating 
time  of  the  equipment  divided  by  the  number  of  failures. 

The  Mean  Time  Between  Failure  is  also  the  reciprocal  of  the 
failure  rate  [21:p.l-2]." 
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The  determination  of  MTBF  in  operational  radars  is 
calculated  using  reported  f 1 ight  hours,  which  in  most 
instances  is  somewhat  less  than  the  total  radar  operating 
time.  A  modifier  factor  (k=1.5),  applied  to  the  MTBF,  is 
used  to  correct  for  the  difference.  Table  9  presents  the 
same  comparison  with  the  field  results  adjusted  by  the 
modifier  factor,  now  defined  R  rate  adjusted.  It  can  be 
seen  that  the  field  results  for  the  radar  are  now  92  per¬ 
cent  of  that  demonstrated  during  reliability  qualification 
testing. 


TABLE  9 

COMPARISON  OF  PREDICTED  RELIABILITY  VERSUS  ADJUSTED 
OPERATIONAL  RESULTS,  APG-663 


LRU 

Production 

Prediction 

RQT 

Results 

Field 

Results 

Antenna 

1764 

370 

182 

LPRF 

651 

148 

225 

Transmitter 

1366 

185 

303 

DSP 

576 

Indeterminate 

599 

Computer 

2008 

738 

449 

System 

187 

57 

52 

aR  Rate  Adjusted,  defined  as  Field  MTBF  multiplied 

by  1.5. 

SOURCE :  see  Table  8 . 
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Analysis  of  Field  Reliability  Parameters.  The 


analysis  in  this  section  deals  with  measureable  reliabil¬ 
ity  parameters  experienced  by  the  APG-66  during  its  deployed 
phase.  The  reported  system  failure  occurrences  and  aircraft 
flight  times  are  extracted  from  data  reported  in  the  RCS 
LOG-MMO(AR)  7170  computerized  output.  Table  15  (Appendix  A) 
presents  the  APG-66  reported  field  data  used  in  the  analysis. 

The  first  analysis  performed  was  a  determination  of 
the  failure  rate  (X)  characteristics  exhibited  by  the  radar 
during  its  deployed  period  ending  January  1981.  The  failure 
rate,  X,  is  defined  to  be  the  inverse  of  the  Mean-Time- 
Between-Failure ,  or  (jjjrgp)  • 

It  is  generally  accepted  that  complex  avionics, 
such  as  radar,  exhibit  a  failure  rate  which,  when  plotted 
as  a  function  of  time,  portray  a  bathtub  shaped  curve 
(Figure  4) .  The  curve  represents  three  theoretical  divi¬ 
sions  of  an  equipment’s  life.  In  reference  to  Figure  4 
these  three  divisions  are  the  bum- in  period,  where 
failures  occur  usually  as  a  result  of  poor  manufacturing 
and  quality-control  techniques  during  the  production  pro¬ 
cess;  useful  life,  where  the  failures  are  usually  due  to 
chance  (as  a  result  of  sudden  stress  accumulations) ;  the 
final  period  is  wearout,  where  failures  occur  due  to  com¬ 
ponent  aging.  At  some  period  of  component  life,  the  early 
failure  rate  of  a  system  will  decrease  and  approach  the 
constant  failure  rate  representative  of  the  useful  life 
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period.  In  that  the  APG-66  radar  is  a  relatively  new  sys¬ 
tem,  it  is  not  improbable  that  its  failure  rate  character¬ 
istics  are  representative  of  the  early  life/normal  life 
juncture . 

Table  16  (Appendix  A)  presents  the  data  used  to 
examine  this  hypothesis.  This  failure  rate  data,  pre¬ 
sented  as  a  scatter  plot  in  Figure  5,  clearly  depicts  the 
exponential  curve  characteristics  identified  with  "early 
life"  in  Figure  4.  The  data  does  not  allow  an  assessment 
of  whether  a  constant  failure  rate  has  been  achieved; 
however,  it  appears  that  the  upward  swing  in  the  curve 
near  the  end  of  the  data  cycle  may  be  due  to  imperfect 
repair  of  substandard  components  (3:71). 

Further  analysis  of  the  APG-66  failure  distribution 
is  depicted  in  Figure  6,  a  graph  which  shows  cumulative 
failures  plotted  against  cumulative  flying  time.  The  non¬ 
linearity  of  the  plot  is  indicative  of  a  non-constant 
failure  rate.  The  slope  of  this  curve  at  any  one  point  is, 
in  fact,  the  failure  rate. 

In  order  to  gain  insight  into  the  reliability 
growth  of  the  system  during  its  deployed  phase,  the  cumula¬ 
tive  AFM  66-1  R  Rate  (reported  flight  hours  divided  by 
failure  occurrences)  was  plotted  against  the  cumulative 
flying  time  incurred  during  the  same  period.  A  scatter- 
plot  of  the  data  in  Figure  7  reveals  a  tendency  towards 
linearity.  As  such,  a  linear  least  squares  regression 
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.  6.  Graph  of  APG-66  Cumulative  Failures 
vs  Cumulative  Flying  Time  (Reporting 
Period:  June  1979  to  January  1981) 


model  was  fit  to  the  data  yielding  a  predictive  equation 
of  y  =  12.552  +  0.00042x  and  a  correlation  coefficient 
equal  to  0.981.  The  slope  of  the  regression  equation, 
0.00042,  is  in  fact  the  reliability  growth  experienced  by 
the  radar  in  operational  use.  That  is,  the  radar  relia¬ 
bility  is  increasing  by  0.4  hours  between  failures  for 
every  1,000  hours  of  operational  flying. 

A  final  analysis  was  performed  to  determine  if 
increased  operational  use  of  the  system  would  increase  the 
system's  reliability.  A  more  effective  time  to  analyze 
this  hypothesis  would  be  after  the  system  had  entered  its 
useful  life  period  referred  to  earlier.  A  scatter  plot  in 
Figure  8  of  monthly  MTBF  versos  monthly  flying  time  again 
shows  a  trend  toward  linearity.  A  least  squares  linear 
regression  model  of  the  data  yields  a  predictive  equation 

A 

of  y  =  6.347  +  O.Ollx  with  a  correlation  coefficient  of 
0.888.  The  positive  slope,  though  it  is  contaminated  by  a 
decreasing  failure  rate  and  learning  curve  effects,  does 
seem  to  indicate  that  increased  frequency  of  system  use 
has  a  positive  effect  on  MTBF. 

APG-66  Field  Maintainability 

Introduction 

This  section  examines  the  maintainability  param¬ 
eters  experienced  by  the  APG-66  in  its  deployed  phase.  The 
analysis  begins  with  an  examination  of  the  distribution 
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Fig.  8.  Graph  of  APG-66  Monthly  MTBF  vs  Monthly  Flying 
Time  (Linear  Equation,  by  regression  y*6.347  + 
O.Ollx;  correlation  coefficient  =  0.888) 
(Reporting  Period:  June  1979  to  January  1981) 
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of  the  ratio  of  maintenance-man-hours  (MMH)  per  failure 
occurrence  (OCC)  data  points  and  closes  with  a  scrutiny  of 
field  maintainability  experience.  APG-66  field  maintain¬ 
ability  is  compared  with  the  predictions  forecast  at  the 
critical  design  review  and  also  with  the  results  attained 
during  the  intermediate  level  maintainability  demonstration. 
The  data  used  in  these  analyses  was  obtained  from  RCS-MMO 
(AR)  7179,  titled  "Summarized  Maintenance  Actions  for 
Selected  Work  Unit  Codes,"  which  is  a  computer  product 
generated  by  the  Air  Force  Logistics  Command  management 
information  system.  In  particular,  the  APG-66  maintenance 
data  extracted  was  valid  for  a  twenty-month  reporting 
period  (June  1979  through  January  1981)  and  included  both 
"on-equipment"  (scheduled  plus  unscheduled)  and  "off- 
equipment"  (shop  repair)  maintenance  repair  times  and 
actions . 

The  two  main  objectives  of  this  section  are  to 
first  determine  if  a  learning  curve  effect  exists  in  the 
maintainability  of  the  APG-66,  and  then  to  determine  the 
applicability  of  critical  design  review  predictions  and 
maintenance  demonstrations  in  forecasting  expected  system 
mean-time-to-repair  (MTTR) . 

Data  Analysis 

Examination  of  Maintenance-Man-Hour  Distribution. 

In  order  to  determine  the  possible  existence  of  a  learning 
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curve  effect  in  the  maintainability  of  the  APG-66,  field 
flight  line  repair  data  (scheduled  plus  unscheduled 
on-equipment  maintenance  repair  actions  and  man-hours) 
was  analy2ed  for  a  twenty-month  period  of  operational 
experience.  Table  17  (Appendix  A)  depicts  the  information 
extracted  from  RCS  LOG-MMO(AR)  7179,  including  radar  units 
in  the  operational  inventory  (INV) ,  maintenance-man-hours 
(MMH)  allocated  during  the  calendar  month,  total  main¬ 
tenance  occurrences  (as  opposed  to  failures)  during  each 
calendar  month,  and  a  final  column  reflecting  a  maintenance- 
man-hour/occurrence  ratio  (MMH/OCC) . 

Each  MMH/OCC  data  point  was  plotted  against  the 
respective  cumulative  flight  time  figure  in  order  to  deter¬ 
mine  if  the  increase  in  system  familiarity  due  to  the  pas¬ 
sage  of  time  (learning  curve  effect  of  the  form  y=axn  where 
n  =  log  b/log  2)  might  cause  a  decrease  in  the  MMH  required 
to  correct  a  system  deficiency.  The  scatter  plot  of  the 
MMH/OCC  data  points  shown  in  Figure  9  clearly  depicts  the 
opposite  of  that  hypothesized;  in  fact,  the  data  reflects  a 
tendency  towards  linearity  with  a  positive  slope.  The 
slope  can  be  interpreted  as  representing  an  average  increase 
in  maintenance-man-hours  devoted  to  each  maintenance  action 
when  there  is  a  corresponding  increase  in  the  cumulative 
system  flight  time.  A  possible  explanation  for  this 
unexpected  result  is  the  unfamiliarity  with  the  system  on 
the  part  of  maintenance  personnel  due  to  new  radar  units 
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being  deployed  in  the  field.  A  definite  increasing  MMH/OCC 
trend  can  be  seen  occurring  at  the  9500  flight  hour  point 
that  corresponds  to  March  1980. 

Figure  10,  a  graph  of  MMH/OCC  plotted  against 
APG-66  inventory,  more  clearly  emphasizes  this  trend.  The 
rapidly  increasing  slope  occurs  at  the  eighty-nine-unit 
point  which,  assuming  a  seventy- two-unit  equipment  (UE) 
wing,  might  indicate  the  activation  of  a  new  squadron  at  a 
geographically  separated  location. 

A  possible  conclusion,  based  on  the  preceding  dis¬ 
cussion,  is  that  the  training  of  new  personnel  in  the 
diagnosis  and  maintenance  of  the  APG-66  appears  to  be  some¬ 
what  deficient.  A  second  possible  explanation  for  the 
anomaly  is  that  APG-66  diagnostic  equipment  may  be  lagging 
the  radar  inventory,  thus  requiring  a  greater  use  of  time- 
consuming  manual  system  testing. 

Predicted  versus  Field  APG-66  Maintainability.  This 
section  analyzes  the  field  maintainability  of  the  APG-66 
in  comparison  with  the  predictions  of  the  critical  design 
review  and  the  test  results  of  the  maintainability  demon¬ 
stration.  The  field  data  used  in  the  analyses  were 
extracted  from  RCS  LOG-MMO(AR)  7179  for  a  six-month  report¬ 
ing  period  (June  1979  through  January  1980) . 

The  first  analysis  examines  the  field  flight-line 
fault  diagnostic  capability  of  the  maintenance  personnel. 
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APG-66  INVENTORY  (IN  RADAR  UNITS  INSTALLED) 


Fig.  10.  Graph  of  On- Equipment  Mean  Maintenance 

Hours  per  Occurrence  vs  APG-66  Inventory 
(Reporting  Period:  June  1979  to  January 
1981) 
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the  associated  test  equipment,  and  the  built-in-test  self¬ 
test  features  of  the  F-16  aircraft.  To  perform  this 
analysis,  it  was  necessary  to  determine  the  number  of  the 
total  maintenance  occurrences  that  were  classified  as 
Type  6  (no  defect)  how-malfunction  codes  for  "on-equipment" 
maintenance.  Table  10  displays  the  percentage  of  unveri¬ 
fied  failures  by  LRU. 


TABLE  10 

FLIGHT-LINE  DIAGNOSTIC  RESULTS,  APG-66 


LRU 

Total 

Occurrences 

Unverified 

Failures 

Percent 

Unverified 

Failures 

Total 

Complexity 

(Parts 

Count) 

Antenna 

169 

14 

8.28 

1653 

LPRF 

172 

21 

12.21 

3172 

Transmitter 

128 

10 

7.81 

2150 

DSP 

76 

13 

17.11 

4588 

Ccnputer 

104 

18 

17.30 

2700 

Control  Panel 

32 

7 

21.88 

245 

System 

681 

83 

12.18 

- 

Reporting  Period:  June  1979  to  January  1981. 


Maintainability  experience  usually  dictates  that 
as  equipment  complexity  increases,  failure  diagnostic 
accuracy  decreases  (3:219) .  Such  is  not  the  case  in  this 
analysis;  in  fact,  the  least  complex  LRU,  the  radar  control 
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panel,  exhibited  the  highest  rate  of  serviceable  diagnosis. 
It  also  appears  that  fault  isolation  in  the  digital  signal 
processor  and  computer  LRUs  is  somewhat  deficient. 

As  was  noted  in  Chapter  IV,  the  radar  subcontractor 
was  very  optimistic  during  the  critical  design  review  of 
the  predicted  maintainability  of  the  APG-66.  Although  not 
achieving  the  predicted  MTTR,  the  APG-66  intermediate  level 
maintainability  demonstration  achieved  impressive  results 
in  minimizing  system  total  repair  time  (consisting  of  fault 
isolation  time,  remove  and  replace  time  and  baseline  test 
time)  . 

The  APG-66  maintainability  experienced  in  the  field, 
however,  has  to  this  point  not  achieved  the  predicted 
results.  In  order  to  determine  the  extent  to  which  pre- 
production  predictions  were  satisfied,  it  was  necessary  to 
determine  the  maintenance-man-hours  expended  for  on- 
equipment  repair  for  the  total  maintenance  actions  recorded. 
This  data  was  available  in  RCS  LOG-MMO (AR)  7179  for  the  six- 
month  period  of  concern,  August  1980  to  January  1981. 

The  MTTR  was  calculated  for  each  APG-66  LRU  by  summing  the 
respective  total  on-equipment  MMHs  expended  and  dividing  by 
the  total  number  of  repair  actions  allocated  to  system 
deficiency  occurrences.  Table  11  presents  these  field 
results  in  comparison  with  the  predictions  of  the  critical 
design  review  and  demonstration  test. 
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TABLE  11 


COMPARISON  OF  ON-EQUIPMENT  MTTR,  CRITICAL  DESIGN 
REVIEW  PREDICTIONS  AND  MAINTAINABILITY 
DEMONSTRATION  VERSUS  OPERATIONAL 
RESULTS,  APG-66 


LRU 

CDR  Predicted3 

MTTR  (Minutes) 

Demonstration3 
MITR  (Hrs/Min) 

Field0 
MTTR  (Hrs) 

Antenna 

20.44 

2  hrs  24  min 

4  hrs  48  min 

LPRF 

42.64 

1  hr  54  min 

4  hrs  12  min 

Transmitter 

86.31 

51  min 

4  hrs  6  min 

DSP 

30.64 

29  min 

4  hrs  12  min 

Radar  Gcrputer 

30.21 

32  min 

3  hrs  30  min 

HOP 

28.26 

10  min 

2  hrs  24  min 

System 

42.05 

1  hr  12  min/ 

0  hrs  58  min 
(adjusted) 

3  hrs  52  min 

aSOURCE 
trol  Radar,  p. 

:  Critical  Design 
10-10. 

Review  (U)  F-16 

Fire  Con- 

SOURCE:  Intermediate  Level  Maintainability  Assess¬ 
ment  and  Demonstration  Report  for  the  F-16  Fire  Control 
Radar . 


CReporting  Period:  August  1980  to  January  1981. 
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Reliability  Comparison,  APG-66  versus  APQ-120 


Objective 

This  section  uses  a  1973  radar  reliability  study, 
"Research  Study  of  Radar  Reliability  and  Its  Impact  on 
Life-Cycle  Costs  for  the  APQ-113 , -114 , -120  and  -144  Radars," 
as  a  foundation  of  reliability  and  maintainability  com¬ 
parison  of  the  APG-66  and  APQ-120  radars.  Some  of  the 
methodologies  and  comparative  analyses  used  in  that  study 
are  employed  in  this  effort  to  standardize  the  results. 

In  addition,  all  data  pertaining  to  the  APQ-120  radar  is 
extracted  from  this  report  and  is  assumed  to  be  valid. 

This  analysis  focuses  on  the  relative  equipment 
capabilities,  a  comparison  of  the  reliability  qualifica¬ 
tion  test  programs,  and  a  generalized  look  at  the  experi¬ 
enced  field  reliabilities.  The  major  objective  of  this 
section  is  to  determine  what,  if  any,  progress  has  been 
made  in  complex  avionics  reliability  development. 

Equipment  Capabilities  and  Relia¬ 
bility  Program  Comparison 

It  is  important  to  realize  in  any  comparative 
analysis  that  different  equipment,  even  though  performing 
similar  functions,  may  have  different  functional  capabili¬ 
ties,  and  that  those  capabilities  may  differ  in  the  degree 
of  technical  performance.  This  effort  does  not  attempt  to 
measure  this  relative  technical  equipment  performance, 
but  rather  focuses  on  the  predicted  and  achieved  reliability 
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maintainability  results.  A  comparison  of  these  func¬ 
tional  complexities  and  capabilities  is  presented  in 
Figure  11.  Although  the  APQ-120  and  APG-66  are  not  of 
similar  design  vintage,  their  functional  capabilities, 
system  complexities  and  field  application  are  very  similar 

Table  12  presents  -i  dramatic  contrast  between  the 
two  radars  in  the  manner  in  which  the  reliability  qualifi¬ 
cation  test  programs  were  conducted.  Whereas  the  APQ-120 
achieved  a  demonstrated  MTBF  of  only  4.3  hours  based  on  a 
contractually  required  forty-five  hours,  the  APG-66 
achieved  a  fifty-seven -hour  MTBF  of  a  seventy-hour  require 
ment.  This  contrast  can  be  attributed  to  several  factors, 
foremost  among  them  is  the  overall  conduct  of  the  relia¬ 
bility  program.  Specifically,  while  the  APQ-120  was  sub¬ 
jected  to  only  ninety-six  relevant  hours  of  reliability 
qualification  test  time,  the  APG-66  underwent  738  hours. 
Strict  enforcement  of  the  contractual  MTBF  at  this  stage 
is  also  important.  This  facet  is  most  evident  considering 
that  the  APG-66  achieved  82  percent  of  its  reliability 
qualification  test  MTBF  requirement,  whereas  the  APQ-120 
was  notably  deficient,  achieving  only  47  percent  of  its 
requirement . 

Field  Reliability/Maintain¬ 
ability  Comparison  "" 

Table  13  compares  the  APQ-120  to  the  APG-66  in 
field  reliability  expressed  in  mean-flight-hours-between 
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RADAR 

COMPLEXITY  (ELECTRICAL  PARTS) 
ELECTRICAL  LRUs 

APQ-120 

13,553 

19 

APG-66 

8,721 

6 

Search 

X 

X 

Detection 

X 

X 

LCOSS  Display 

X 

X 

Tracking 

X 

X 

Missile  Illumination 

X 

X 

AOJ 

X 

X 

Air  Combat  Mode 

X 

Mapping 

X 

X 

Ranging 

X 

X 

Expanded  Display 

X 

Freeze  Mode 

X 

DBS 

X 

CRT  Display 

X 

X 

BITE 

X 

X 

Self-Test 

X 

Computation 

X 

X 

Beacon  Capability 

X 

X 

Auto  Photography 

X 

ECM  Features 

X 

X 

EO  Mode 

X 

X 

Fault  List 

X 

Sea  Mode 

X 

Target  History 

X 

Fig.  11.  Equipment  Capability  Comparison, 

APQ-120  vs  APG-66 

aSOURCE:  Research  Study  of  Radar  Reliability  and 
Its  Impact  on  Life-Cycle  Costs  for  the  APQ-113,-114,-120, 
and  -144  Radars,  p.  12. 

bSOURCE:  User's  Manual  (Computer  Program)  for  the 
F-16  FCR. 
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TABLE  12 


RELIABILITY  PROGRAM  COMPARISON 


APG-663 

APQ-120b 

Equipment  Complexity  (parts 
count) 

15,200 

13,500 

Reliability  Test  Specifica¬ 
tion 

MIL- STD- 
7818,  Test 

Plan  III, 

Level  F 

MIL-STD- 
781A, 
Level  A 

Reliability  Test  Timing 

Late 

Late 

Relevant  Reliability  Qualifi¬ 
cation  Test  Hours 

738  Hours 

96  Hours 

LRU  Environmental  Screening 

100  Percent 

0  Percent 

MTBF  Required 

70  Hours 

9  Hours 

MTBF  Predicted 

187  Hours 

45  Hours 

MTBF  Demonstrated 

57  Hours 

4.3  Hours 

aSOURCE:  Final  Report 

on  the  Reliability 

Qualifies- 

tion  Test  of  the  F-16  Fire  Control  Radar. 

^SOURCE:  Research  Study  of  Radar  Reliability  and 
Its  Impact  on  Life-Cycle  Costs  for  the  APQ-113 ,-114 ,-120, 
and  -144  Radars. 


TABLE  13 


AFM  66-1  DATA,  APG-66  VERSUS  APQ-120 


Parameter 

APQ-120a 

APG-66b 

APG-66 /APQ-120 
Ratio 

R  Rate  (hours) 

11 

35 

3.18 

M  Rate  (hours) 

4 

11 

2.75 

Flight  Hours/MMHc 

0.4 

1.36 

3.4 

aSOURCE:  Research  Study  of  Radar  Reliability  and 
Its  Impact  on  Life-Cycle  Costs  for  the  APQ-113 , -114 , -120 
and  -144  Radars,  p.  2-11.  (Reporting  Period:  June  to 
November  1971) . 

b SOURCE ;  RCS  LOG-MMO (AR)  7170.  (Repeating  Period: 
August  1980  to  January  1981) . 

CMMH  includes  scheduled/unscheduled  maintenance  in 
the  field,  shop  and  depot. 

failures  (R  rate)  and  in  field  maintainability  expressed 
in  mean-flight-hours-between-maintenance-actions  (M  rate) . 
The  column  headed  Ratio  to  APQ-120  clearly  shows  the 
APG-66  to  be  a  superior  radar  in  the  reliability  sense. 

It  has  previously  been  noted  that  reliability 
growth  was  considered  an  important  facet  of  the  APG-66 
program.  Figure  12  presents  a  comparison  of  reliability 
history  of  the  APQ-120  (reporting  period  1967  to  1971) 
versus  the  APG-66  (reporting  period  June  1979  to  January 
1981).  It's  evident  from  the  slope  of  the  older  generation 
radar's  curve  that  little,  if  any,  reliability  growth 
occurred.  In  contrast,  as  noted  earlier  in  this  chapter, 
the  APG-66  is  experiencing  an  aggressive  reliability  growth. 
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■Reporting  Period:  June  1979  to  January  1981. 

“Reporting  Period:  1967  to  1971. 

SOURCE:  Research  Study  of  Radar  Reliability  and  Its  Impact  on  Life-Cycle 
Costs  for  the  APQ-113 , -114 ,-120 ,  and  -144  Radars,  p.  225. 


Table  14  compares  the  maintainability  aspects  of 
the  fire  control  radars  in  terms  of  maintenance -man-hours 
per  maintenance  action  (MMH/MA) ,  a  figure  that  reflects 
greatly  on  a  weapon  system's  availability.  Clearly, 
again,  the  APG-66  is  a  superior  radar  in  this  sense  with 
a  low  average  6.0  MMH/MA.  This  contradicts  the  indication 
that  a  maintenance  learning  curve  for  the  APG-66  did  not 
exist.  A  possible  explanation  may  be  that  the  APG-66  is 
extensively  partitioned  into  six  line-replaceable  units 
(LRUs)  and  eighty-seven  shop-replaceable  units  (SRUs) . 

As  noted  in  Chapter  IV,  considerable  effort  was  devoted  to 
LRU  and  SRU  accessibility  and  testability,  and  it  appears 
the  effort  has  met  with  considerable  success. 


TABLE  14 

UNSCHEDULED  MAINTENANCE  MAN-HOURS  (FLIGHT-LINE)  PER 
MAINTENANCE  ACTION,  APG-66  VERSUS  APQ-120 


Radar 

Unscheduled 
Maintenance  , 
Man-hours (X10J) 

Number  of 
Maintenance 
Actions 

MMH/MA 

Ratio  to 
APQ-120 

APQ-120a 

225 

28,400 

9.0 

1.0 

APG-66b 

9.0 

1,491 

6.0 

0.67 

aSOURCE:  Research  Study  of  Radar  Reliability  and 
Its  Impact  on  Life-Cycle  Costs  for  the  APQ-113-114 ,-120, 
and  -144  Radars,  p.  213.  (Reporting  Period:  June  to 
November  1971). 

bSOURCE:  RCS  LOG-MMO ( AR)  7170.  (Reporting  Period: 
August  1980  to  January  1981)  . 
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Considering  the  many  facets  that  impact  main¬ 
tenance:  equipment  utilization,  maintenance  approach,  man¬ 
power  familiarity  and  training,  among  others,  the  APG-66, 
despite  its  early  point  in  system  maturity,  fares  well  when 
compared  against  the  APQ-120. 
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CHAPTER  VI 


FINDINGS,  CONCLUSIONS  AND  RECOMMENDATIONS 

Introduction 

This  thesis  effort  examined,  as  a  case  study,  the 
field  reliability  and  maintainability  of  the  APG-66  fire 
control  radar  in  comparison  with  its  contractual  require¬ 
ments,  its  pre-production  development  and  test  results 
and  with  the  development  and  performance  of  an  earlier 
generation  radar  (APQ-120) . 

In  order  to  properly  evaluate  the  effectiveness 
of  the  APG-66  subcontractor's  reliability  and  maintain¬ 
ability  programs,  it  is  necessary  to  form  qualitative  as 
well  as  quantitative  estimations  of  program  performance. 

In  addition,  the  conclusions  that  follow  have 
several  obvious  limitations.  Foremost  is  the  fact  that 
the  APG-66,  in  its  earliest  phase  of  operational  maturity, 
has  not  yet  exhibited  the  period  of  constant  failure  that 
is  typical  of  an  equipment's  useful  life.  This  factor 
pessimistically  affects  the  quantitative  assessment  of  the 
APG-66  field  reliability  performance.  The  nature  of  the 
comparison  of  the  APG-66  with  the  APQ-120  radar  is  the 
second  limitation.  This  comparison  provides  only  an 


evolutionary  examination  of  reliability  programs  and  does 
not  attempt  to  compare  the  relative  technical  capabilities 
of  the  systems. 


Findings  and  Conclusions 

Reliability 

The  reliability  predictions  presented  during  the 
F-16  fire  control  radar  critical  design  review  were  very 
optimistic  in  comparison  with  the  operational  performance 
experienced  to  date.  Some  of  this  optimism  is  no  doubt 
founded  in  the  subcontractor's  propensity  to  overstate  the 
radar's  capabilities.  Other  factors  may  contribute  to  this 
situation,  however,  such  as  substandard  parts  performance 
and  an  equipment  environment  different  than  that  forecast. 

The  reliability  qualification  test  was  an  impor¬ 
tant  phase  in  the  development  of  the  APG-66.  The  fact  that 
the  radar  achieved  95  percent  of  it^>  .  "rget  MTBF  can  be 
atrributed  to  both  the  high  level  at  agement  emphasis 
on  reliability  and  the  strict  enforcement  of  contractual 
test  requirements.  The  test  results  proved  to  be  an 
accurate  prediction  of  initial  field  reliability  perform¬ 
ance  (both  MTBF  and  reliability  growth)  which  indicates 
the  testing  environment  was  representative  of  the  actual 
field  environment.  As  observed  in  the  comparison  with  the 
APQ-120,  the  test  function  in  weapon  systems  acquisition  is 
a  primary  determinant  of  equipment  field  performance. 
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The  decreasing  APG-66  failure  rate,  determined 
according  to  reported  field  data,  is  asymptotically  approach¬ 
ing  a  constant  figure  representative  of  the  radar's  useful 
life.  It  is  at  this  juncture  where  intense  and  concen¬ 
trated  management  emphasis  on  the  system's  enhancement 
will  contribute  significantly  to  final  reliability  perform¬ 
ance.  This  is  evident  in  the  positive  reliability  growth 
experienced  to  date. 

Maintainability 

The  APG-66  maintainability  analysis  resulted  in 
two  general  findings.  The  first  is  that  the  APG-66  critical 
design  review  and  the  intermediate  level  maintainability 
demonstration  predictions  were  extremely  optimistic  of 
forecast  field  maintainability.  Although  the  maintain¬ 
ability  concept  of  the  radar  received  a  significant  degree 
of  management  and  design  emphasis,  the  reported  field 
experience  indicates  that  actual  repair  times  exceed  demon¬ 
stration  predictions  by  400  percent.  When  compared  with 
the  APQ-120,  however,  maintainability  figures  indicate 
that  a  good  deal  of  progress  has  been  made  in  this  area 
with  the  APG-66  fire  control  radar. 

The  second  finding  is  that  the  reported  field  main¬ 
tainability  data  suggests  that  a  maintenance  learning 
curve  may  not  exist.  Conversely,  the  figures  indicate 
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the  ratio  of  maintenance-man-hours  per  failure  occurrence 
to  be  increasing  as  cumulative  flight  time  increases. 

These  two  findings  may  be  due  to  any  of  several 
factors.  One  factor  may  be  a  deficiency  in  training  the 
personnel  who  maintain  the  system.  A  second  possible 
explanation  is  that  the  field  deployment  of  diagnostic 
equipment  may  be  lagging  the  radar  inventory.  These  condi¬ 
tions  are  somewhat  to  be  expected,  and  only  future  analysis 
will  reveal  true  APG-66  maintainability. 

Recommendations 

The  two  major  recommendations  as  a  result  of  this 
thesis  effort  are  that  systems  program  office  personnel  con¬ 
tinue  to  enforce  strict  compliance  of  contractual  efforts; 
these  efforts  must  be  demonstrated  to  program  officials' 
satisfaction  through  intense  and  thorough  system  testing. 

In  view  of  the  research  findings  and  conclusions, 
an  expansion  and  continuation  of  this  case  study  is  recom¬ 
mended.  In  the  future,  APG-66  reliability  and  maintain¬ 
ability  field  experience  should  be  more  representative  of 
the  performance  typical  of  a  mature  system. 
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APPENDIX  A 
REPORTED  FIELD  DATA 


TABLE  15 


APG-66  REPORTED  FIELD  DATA 


Year 

Month 

Flight  Time 

Failures 

79 

June 

430 

41 

July 

474 

27 

August 

479 

36 

September 

779 

56 

October 

1095 

48 

November 

1069 

67 

December 

955 

103 

80 

January 

1161 

61 

February 

1424 

77 

March 

1589 

75 

April 

1783 

85 

May 

2370 

68 

June 

2048 

56 

July 

2366 

69 

August 

2493 

82 

September 

2276 

65 

October 

3269 

61 

November 

2931 

92 

December 

2805 

87 

81 

January 

3002 

93 

June 

SOURCE:  RCS  LOG-MMO (AR) 
1979  to  January  1981. 

7170.  Reporting  Period: 
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TABLE  16 


APG-66  FAILURE  RATE  DISTRIBUTION 


*  (Monthly  MTBf) 

Cumulative 
Flight  Time 

.095 

430 

.057 

904 

.075 

1383 

.072 

2162 

.044 

3257 

.063 

4326 

.108 

5281 

.053 

6442 

.054 

7866 

.047 

9455 

.048 

11238 

.029 

13608 

.027 

15656 

.029 

18022 

.033 

20515 

.029 

22791 

.019 

26060 

.031 

28991 

.031 

31796 

.031 

34798 

SOURCE:  RCS  LOG-MMO(AR)  7170.  Reporting  Period: 
June  1979  to  January  1981. 


TABLE  17 


APG- 

-66  MMH, 

INVENTORY 

AND  FAILURE 

OCCURRENCES 

Month/Yr 

Inv 

MMH 

Total 

Occurrences 

Total 

MMHvOccurrences 

Jun  1979 

21 

319 

118 

2.70 

Jul 

27 

311 

74 

4.20 

Aug 

33 

370 

149 

2.48 

Sep 

39 

399 

185 

2.17 

Oct 

47 

487 

173 

2.82 

Nov 

58 

627 

207 

3.03 

Dec 

63 

985 

370 

2.66 

Jan  1980 

72 

777 

202 

3.85 

Feb 

79 

751 

253 

2.97 

Mar 

89 

1070 

385 

2.78 

Apr 

96 

1194 

334 

3.57 

May 

110 

1363 

308 

4.43 

Jun 

117 

1064 

198 

5.37 

Jul 

121 

1504 

310 

4.85 

Aug 

125 

1685 

301 

5.60 

Sep 

135 

1230 

203 

6.06 

Oct 

145 

1253 

212 

5.91 

Nov 

157 

1623 

263 

6.17 

Dec 

173 

1730 

268 

6.46 

Jan  1981 

158 

1507 

244 

6.18 

SOURCE:  RCS  LOG-MMO (AR)  7170  (includes  scheduled 
and  unscheduled  "on-equipment"  maintenance) .  Reporting 
Period:  June  1979  to  January  1981. 
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ACF 


AFLC 

AFSC 

AFSC 

AIS 

AOJ 

BIT 

BITE 

CDR 

CFE 

DBS 

Demo 

DOD 

DSARC 

DSP 

DT&E 

ECM 

EO 

FCR 

FSD 

GAO 

ILMD 


Air  Combat  Fighter 

Air  Force  Logistics  Command 

Air  Force  Specialty  Code 

Air  Force  Systems  Command 

Avionics  Intermediate  Shop 

Acquisition-On- Jam 

Built-In-Test 

Built- In-Test  Equipment 

Critical  Design  Review 

Contractor  Furnished  Equipment 

Doppler  Beam  Sharpened 

Demonstration 

Department  of  Defense 

Defense  Systems  Acquisition 
Review  Council 

Digital  Signal  Processor 

Development  Test  and  Evaluation 

Electronic  Counter  Measures 

Electro-Optical 

Fire  Control  Radar 

Full-Scale  Development 

General  Accounting  Office 

Intermediate  Level  Maintainability 
Demonstration 


INV 


Inventory 


X 


Failure  Rate 


LCC 

Life-Cycle  Cost 

LCOSS 

Lead  Confuting  Optical  Sight 
System 

LPRF 

Low-Power -Radio-Frequency 

LRU 

Line  Replaceable  Unit 

MA 

Maintenance  Actions 

M  ^ 

Ct 

Corrective  Maintenance  Time  to 
Repair  an  LRU 

MFL 

Maintenance  Fault  List 

MIL-HDBK 

Military  Handbook 

MIL-STD 

Military  Standard 

MMH 

Maintenance  Man-Hours 

M  Rate 

Mean  Flight  Hours  Between  Main¬ 
tenance  Actions 

MTBF 

Mean  Time  Between  Failures 

MTTR 

Mean  Time  to  Repair 

O&S 

Operations  and  Support 

OCC 

Failure  Occurrence 

RADC 

Rome  Air  Development  Center 

RAF 

Royal  Air  Force 

RCP 

Radar  Control  Panel 

RCS  LOG-MMO (AR) 

7170 

Maintenance  Actions,  Man-hours, 
and  Aborts  by  Work  Unit  Code 

RCS  LOG-MMO (AR) 

7179 

Summarized  Maintenance  Actions 
for  Selected  Work  Unit  Codes 

ROC 

Required  Operational  Capability 

RQT 

Reliability  Qualification  Testing 
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R  Rate 


SPO 

SRLJ 

TAC 

UE 

XMTR 


Reported  Flight  Hours  Divided  by 
Failure  Occurrences 

Systems  Program  Office 

Shop  Replaceable  Unit 

Tactical  Air  Command 

Unit  Equipment 

Transmitter 
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